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Foreword 



An literature references in this book are "upibered and 
each'work is referred to- by its appropriate number in paren- 
thesire.g! "Anderson (59) contends. . .". If several refe- 
rences are listed, the numbers are Separated by comma or 
ampersand, e.g. "Several authors 3, S. 27 & 39) report... . 
When page numbers are given, they are separated from the 
referencl humber by colon, e.g. "The authors 's discussion 
•Sf tMs point {45:1 2f )...". Thus. (1:2) refers to reference 
•no. 1. page 2. whereas (1.2) refers to references no. 1 and 
2. ,. 

The appendices found in this book -"are giWen in Swedish", be- 
cause"^ l|?onsider it very important ^hat^the exact joring 
of.testl. instructions, p-rogram frames etc is made avail- 
able tQ the reader. This will, of course, be so|"^what in- 

- convenient to the reader who has no Swedish. I will, how- 
e!er! blhappy tp provide any non-Swedish-speakmg reader 

-i reqi^sti ng.it with an English translation of these appen- 
dices. 



The study presented here originated in the then National 
School for Educational Research in Linkofjing. fjch was al- . 
so the institution where I started my work in the field of 
educational research. For this study as well as for a great 
part of my. professional development -I am. then, very much . 
indebted to the National school for Educ^Honal Research, o 
whirh for a ten year period, under the ejfltliysiastic Ifeader- 
sh p of rSf of^Je'^Malmquist. constitlJt|d a highly v tal 
centre of 'research and development withiH^Swedish education. 
ForSnately, I have enjoyed, eyen-after the closing do.vm of. 
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the National School for Educational Research, the encoura- 
gement and support of Eve Malmquist, as a colleague, super- 
visor and friendr . • & - - 

r^am much indebted to Mrs. Noomi Swedberg for her conscien- 
tious typing and observant proof-reading. 

To all others, no one mentioned and no one forgotten, who 

helped in carrying out the work reported here, I am also 
grateful.^ 

Linkoping, July, 1975. H. U. G. 
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Introduction 



PVoarateed instruction (PI) has attracted considerable in-. 
teJesHrthe field of. education for some twenty years now. 
The literature is rich in theoretiaal discussions and inves- 
ti^at?ion results as well as in practical- guidance for pro- 
gram construction. Sevet-al different t^I "jode^^J^^"^^""" . 
proposed, discussed and tested, but by a^t€ large the Skin- 
Syrian model seems to be the tjne most commonly accepted and 
SdoJtSd. In fac*. many ^eem to believe that th^ rationale . , 
proposed by Skinner is the only sound basis for PI. 

Variations of the Skinner model have t>een' suggested, some 
of them by Skinner himself, but it is nevertheless Doss^ b e 
to discern somd basic features of what we today might call 
traditional PI. The instructional unit, the 'frame . in a _ 
Skinnerian program is, characterized by 
- - Ehe deSaSd for oven, student-constructed responses and 
- information about the correct? response immediately after, 
each unit (cf. 120 and 121). 

<^ • '"-It 

Numerous attempts have been made to evaluate the learning 
•SIS of Sese features: Several alternative response mo- 
5e have blfen t^kted and compared, and the differential ef- 
fects of various amounts or types of Infonnation about the- 
coSec? responses have alsp been ^Stivesti gated. However. so 
•far none of the many proposed alternative PI models has 
been proven-general ly and unambiguously superior .to the 
StSer?. It has eyen been argued that. such comparisons are 
rather futile. As one researcher al aims, after havi-ng re- 
'viewed ex'Derim.ental results in this field: ^ . > 
«f am SeSared to. stick out my neck and suggest that some 
gains so far obtained by the'use of teaching machines 
and progranmies have-been primarily due to the more.effl- 
cient use of pupil time." (62:435). 

On the other hand, many studies have demonstrated, beyond 
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any reasonable cU)ubt,,that the Skinnerian response mode in 
.Pi is particularly time-consuming, which may make it less ' 
effic-ient than other models, not demanding overt, student-' 
constructed responses. 

It has also been ar'ifued that the response mode effect .in PI 
is dependent upon a number of different program feind student 
characteristics. And similar relationships may well hold 
for variations in the information about the correct respon- 
ses. ; * ^ 

Skinner's PI model is based* on a particular learning theory 
Whether this theory is acceptable or not is a e^uestion far 
beyond the scope of this* study. It should be noted, though, 
that this learning theory has been more and more intensely 
•challenged in recent years. Regardless of how one judges 
Skinner's learning theory, there are, howeyer, practical 
reasons for questioning the efficiency and viability of his 
PI model, particularly as far as its application in school 
instruction is croncerned. 



Terminology 



"Arv adequate definition permits us always to. tell when a 
sentence containing the defined term is true..." (72:48). 

^ "In general, definition in science- stops when all descrip- 
' ' tiye termi in^.the definition refer either to physical 
oWects or to some directly obser^vable properties and re- 
lations of and among them." (72:51). 

Definitions of terms are often necessary in order to ensure 
that propositions containing the- terms in question have a 
clear meaning* Hjotwever, such definitions are no end in them- 
selves. Thei'r purpose isl to clarify concepts which do not 
already have a commonly accepted^ denotation 9 and- this is a- 
'chieved by relating them to'concepts which do have commonly , 
accepted denotations. Definition is also needed when the com-^ 
moryly^ accepted denotation of ai term can be regarded as ambi- 
gujius or misleading. 

Since'PI is a' comparatively new and fast developing field, 
it is only natural that we have not attained within it a 
very high level of tenrnriological stability ahd precision. - 
It S^ems, therefore, necessary to start this study of respon- 
se mode and response infomiation issues in PI with a discus- 
sion of certain t'erminological questions. The following dis- 
cussion concerns only terms which, in my opinion, are used 
without sufficient clarity of concept. Tenfis not discussed 
in this chapter are consequently assumed to have commonly 
accepted - a^nd Appropriate - denotations. 



2.1 ' ' PROGRAM VARIABLES AND STUDENT BEHAVIOUR 

PI investigations often have the aim of studying program va^^ 
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riables (or programing variables; see 90). Such variables 
are of Interest because \Jhey are expected to determine or 
influencfe^ student behavioiTK^and, consequently, the effect 
of programed instruction. """^^^ ^ 

Since student behaviour is not determined' s^olely by program 
variables, it is important to make a clear distinction bet- 
ween the concept 'program variable' ^ i,e. a* variable pro- 
gram property, defined independently of istudent behaviour, 
and what might be called 'PI variable' ^ i.e,.. a variable stu- 
{ient behaviour property,^ which assumedlyjs a consequence of 
an interaction between program and student. It should be 
noted that the concept 'program' here inxriudes all instruc- 
tions given v/ith the program, ^ 

Clearly, program variables as defined above are suitable . 
i ndependent variables in PI studies^ since such 
variables can be direct^ manipul^^teC. PI variables, on the 
other hand,* are in a sense alvjays dependent variables, as 
they cannot be directly manipulated by the experimentator or 
the programmer. With the definition of program variable pro- 
posed here, it seems necessary to chl^ose terms for various 
program variables so that "they dp not imply any assumptions 
or expectations concerning stu^lent behaviour. This is espe- 
cially important, when the existence of a relai,tionship - or 
the degree of this relationj^ip - betvyeen a program charac- 
teristic and student behaviour cannot be controlled. 

Examples of terminology tending to obscure the difference 
between program chara€t6ristics and what is here called PI 
variables are not infrequent in the PI literature. This was. 
pointed out several wears ago by Juel & Metfessel (51). One 
of the main objective^ of the present discussion of PI ter- 
minology is to reveal and eliminate such sources of confu- 
sion. 



2.2 RESPONSE INFORmTIDN 



Skinnerian PI is characterized by, among other things, the 
student's being informed, immediately upon completion 



of eaeh program unit, about the correct response, which is 
presented in the form of« a key answer (121:99). Such presen- 
tation in full of the correct response can be labelled r e - 
sporTse information (cf . 56) or, more prer 
cisely, key answer presentation. These 
two terms are neutral in relation to different theories or 
assumptions about the effect of the presentation of the 
correct responses. In this respect they differ from such 
commonly used terms as ''confirmation of responses*, rein- 
forcement' and 'feedback'. The term 'response information 
denotes a program variable, since the type and frequency Qf 
such information can vary independently of student charac- 
teristics and behaviour. The terms 'confirmation', rein- 
forcement' and 'feedback', however, are not suitable fo> 
the denotation of program variables.^ 

Confirmation of response implies that 
a correct student response has been given, otherwise only 
correction is possible: It also implies that the student 
compares his own response with the correct response » and 
that he is aware of the correctness of his own response. 
There is, of course, the possibility of confirming an in- 
correct response, if the student fails to notice some dif- 
ference between his own response and the correct response. 
In any case, the occurance of response confirmation is' de- 
pendent upon the student's behaviour. y> 

Reinforcement imp;iies, if it is Uken to mean 
positive reinforcement, that confirmation of response has 
occurred and that, as a consequence of this confi motion, 
the probability of the student's responding correctly the 
next time is increased.' Thus, reinforcement ^ even more so 
than confirmatior, is dependent upon factors 'within' the 
student. It has been argued, for instance by Kay (93), that 
confirmation always implies, reinfarcement. But Kay himself 

emphasizes that , , ^ 

"we do not know the conditions under which knowledge 

of results attains and maintains reinforcing proper- 
ties" (93:30). , 

Feedback does not necessarily imply that the^student 
response is correct. 3ut it does imply some comparison bet- 
ween the student response and the correct response. Further- 
more, as feedback in a literal sense presupposes an inter- 
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action tletv,/een student behaviour and 'program behaviour', 
feedback is impossible .tn PI unless an advanced technical 
equipment is used, felhen programnad books, or simple machines 
are used, one can, at most, expect some kind of auto-feed- 
back, i.e. the student may use the response infohnation to 
adjust his response behaviour. 

The program property k e y answer presenta- 
tion can, in interaction with different kinds of student 
behaviour, lead to response confimfation, to reinforcement , 
to feedback, or to neither of these alternatives. Obvious- 
ly, our teminology should reflect this fact and distinguish 
clearly between the objective program property, *on the one 
hand, and its more or less probable effects in a PI situa- 
tion, on the other. Basically the same point has been made 
by Annet: 

"It is suggested that ^reinforcement', , should not be 
used arbitrarily as a substitute for other related terms 
as 'knowledge of results'" (61:282), 
and by Lumsdaine: 

"• .inmiedtats^H^eedback providing confirmation or correc- ^ 
tion .. cannot be equivalated theoretically to any 
simple conception of reinforcement,," (100:86), 

Summarizing this discussion we find that terms, such as 're- 
. sponse infonnation* and 'key answer presentation' are bet- 
ter suited than the others discussed here to denote an in- 
structional variable, which is to be the object of experi- 
mental study. It should be noted, also, that any attempt to 
assess reinforcement or feedback effects in PI is meaning- 
ful only if the presentation of the correct response can be 
shown to have an effect. i. 

Key answer presentation can' vary as to the proportion of 
program units (frames) after which the key answers are given. 
This proportion, expressed in per cent of the total number 
of frames, will in the present study be labelled k e y a n- 
swer frequency. 

A response infonnation schedule where key answers are given 
aftfer some frames only, e.g. every second or third frame, 
is sometimes called 'partial reinforcement' (e.g. in 3 & 
30). This term is inappropriate, since it may,. wrongly, be 
regarded as a counterpart in PI to the much investigated 



partial reinforcement paradigm in animal learning experi- 
ments. An example of this m^isinterpretation is found in 
' Krumboltz & Weisman (32), w^o avoid the term 'rernforcement 
in their report title but nevertheless contend that 

"providing confirming answers on an intermittent scale 
may increase retention in th,e same manner 
(my italits) that interfnittent reinforcement prolongs 
extinction..". 

: As Holland (90:91) has pointed^ out, this is hardly so. The 
' crucial difference is that in traditional, animal learning 
experiments one and t h e same response is alternately, 
reinforced and nof reinforfieji, while in PI partial, or rath- 
er intermittent, response tnformation mean^ that, ^ 
group of different responses, some are followed by 
key answers and some' are not. Even those using the .term 
•partial reinforcement* in tne sense criticised here some- 
times show that they are aware of the inappropriateness of 
the term'. In a report entitled "The effect of partial re- 
inforcement" Berglund states: .... 
".. it is probably inaccurate to speak of partial rein- 
forcement in the /traditional sense of the term in con- 
nection with programmed instruction" (3:2). 
As a general term denoting various key answer frequencies 
less than 100 but greater than 0 per Cent the term 'in- ^ 
termittent responseinformation 
will be used in this study. 



2.3 RESPONSE, RESPONSE MODE AND RESPONSE REQUIREMENT 

•The term 'response' is used in this study as a common label 
for student reactions that are answers to direct or indirect 
questions, or solutions to problems given, or other analo- 
gous student reactions. A response need not be observable 
to anyone except the student himself. 

Many studies of program variables published during the 
1960*s are studies of what is usually called response mode. 
The term 'response mode' does, however, not denote a program 
variable; it is rather a PI variable. The corresponding pro- 




gram variable is the feature of the progr^am (instruction 
etc.) intended to produce the desired student ^response mode, 
i.e. the-.r e s p o n s e r e q u i r e m e^n t pf the pro- 
gram. The distinction between response mode and response re- 
quirement may. seem subtle. Where a specific response requi- 
rement produces, with great probability, the desired respon- 
% se mode, and where one carf continually control that this is 
so, the distinction is in a way superfluous^ and the two 
terms can be regarded as practically interchangeable. This 
may be the case when an overt, observable response is expli- 
citly required and means for recording the response are 
a.vailable. 

If» on the other hand> observable responses are not requi- 
red, there is no immediate way of controlling the corres- 
pondence between response requirenent and response mode. 
Also, the risk that the response is omitted is likely to 
increase. In this case the distinction between required and 
given response is important, since one and the same response 
requlhement may lead to different response modes ^ with small 
possibility of tellipg which response mode each student uses. 
Generally speaking we can study the program variable respon- 
se requirement, w^reas the response mode sometimes must be 
regarded as a hypothetical variable. In this study, the 
term 'response mode', therefore denotes student behaviourp 
and the term 'response requirement' is used to denote pro- 
gram properties. 

Several different terms are used in the PI literature to 
label different response requirements or response modes « The 
following terms will "be discussed here: 

- ^active* versus 'passive' response^ 

- *overt' versus 'covert' pr 'no' response-j 

- 'explicit' versus 'implicit' responses and 

- 'student constructed' versus 'multiple choice' response. 

The terms 'active' and 'p a s s i v e' response are 
good examples of terminological imprecisiotn. 'Active, respon- 
se' is in a sense a tautology, as a response always i^ some 
kind of action, physical or mental. From this point of view 
'passive response' is an impossible temv. The term 'active 
response*' is sometimes used with the implication that the 
response activity in question is observable. Klaus, for in- 
stance, contends that the first rule of programming is to 
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"require active responding" (95:43). 'Active responding is 
then synonymous to the more common term overt responding 
and consequently superfluous. On the vvhole, the terms ac- 
tive' -and 'passive' must be regarded as inappropriate for 
the classification of responses in PI. 

The tenns-'o v e r V4^c o v e r f and 'n o' response-ob- 
viously denote aspects of , response mode. As has already 
been mentioned, the correspondence between response requi- 
rement and response mode can easily be checked when overt 
responses are required, but not when "vert." non-ob^ervabU 
responses -are required. -For response modes labelled no re-' 
sponse' this correspondence is als-o difficult to check, as 
covert responses cannot be distinguished from no respon- 
ses' through observation of student behaviour., Furthermore, 
the 'no response' mode can be a natural consequence of se- 
vef^l different program properties: no, ques-tion has been 
asked and no problem posedi or no instruction to answer the 
question has been given; or instruction has been giv^n not 
to answer the question. - , ' 

Empirical studies with the aim of comparing the effects of 
overt versus covert responses etc. are In most casfes stu- 
dies of the effects of the corresponding response require-^ 
ments. It seems logical that the terminology ' used in such 
studies should make this clear. 

The terms 'e x p 1 i c i t' and 'i m p 1 i c i t' are"_ gene- 
rally used as attributes to 'response (mode)'. Lumsdaine. 
for instance, says: . , ... 

" an explicit response means a' response given by the 
student on some specific occasion identified to him 
through instructions.. - - - ..i m p 1 i c i t respon- 
ses (of a covert sort) .. are responses for which the 
occasion is less clearly identified, such *s subvocal 
.. responses that occur in silent '"eading. in listening 
to a presentation, or in creative thinking' (98:486f). 
Although both 'explicit' and 'implicit' stand « ft^^ibute 
to 'response', Lumsdaine's definition implies that expli- 
cit' characterizes the response requirement, whereas im- 
Dlicit' seems to characterize both response requirement and 
response mode. The expression "an explicit response must 
obviously be taken to mean 'an explicitly required respon- 
se' Even Lumsdaine. who- is obviously trying to remedy the 



terminological confusion, falls to distinguish between re- 
sponse mode and- response requirement. 

, ^ , ,1 

Response requirements are always explicit, since an 'impli- 
\ c^t response Is not required by the.progf9m, but the re- 
sult of some Interaction between student and progr,am. For 
practical purposes it seems reasonable to let 'explicit re- 
sponse requirement' denote any program feature that express- 
edly and uneiqui vocally instructs the student to respond. 
V. This is also in agreement with Lumsdaine's definition (see 
quotation above). AsLumsdaine points out, it is essential 
that thedimension 'explicit - implicit' is not mixed up 
with the dimension 'ovfert - covert - no' response. 

"..the term. 'Implicit responseVhas frequently been used 
to embrace both explicW and implicit covert responses-. 

Explicit responses may be either overt or covert.. 

- - - The further distinction to be made' is that between 
e X p 1 1 c 1.,t covert responses .. and implicit re- 
sponses of a covert sort." (98:487) 

In principle, it is possible that a 'no response' mode is 
either explicit or Implicit. The difference between the two " 
modes is most easily expressed as the difference between de- 
mand^^or n o response, on one hand, and n o demand for re- 
sponse^ on the other. In spite of its subtleness the di- 
stinction is of interest, mainly due to the difficulty of 
distinguishing, In. empirical studies, non-responses from .co- 
vert responses.. TermtnOlogical confusion may arise here if 
student responses are discussed one-sidedly ^in terms of re- 
^ ^sponse m 0 d e, or if a proper distinction is mt made bet- 
ween y^esponse requirement aspects and response mode aspects. 

Lumsdaine (98) labels 'implicit responj^^' a condition 
knov/n from many studies, which cons>^ of letting the stu- 
dents go through a program with the. correct responses writ-c 
ten in all the blanks - a conditiop often call^id 'reading'. 
The same conditionals called "covert ^or rather n o respond- 
ing t§i Wit'trock (55s my italics). In many studies it Is im- 
possible to tell which mode of responding the students 
have used, and the response requirement is often 
Incompletely described. ^An exact rendering of the instruc- 
tion given to the students is, for example, seldom found in 
the published reports. 
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- In cases?.where the original respoijse requirement calls for 
filling l^n blanks, in the program "frames, the 'reading' mode 
has two main variants: 

- response terms are not underlined or in any other way 
emphasized; and 

- response terms are underlined or emphasized by means of 
different colpur, etc, ' 

In the first variant there will probably be no responding, 
whereas- the second would be expected to yield what lums-- 
daine calls "implicit, covert responses". To label the con- 
dition 'reading' has the advantage of not presupposing any- 
thing concerning student behaviour th^t caanot be checked. 

^ In the terms discussed here, this condition can only be de- 
fined negatively, as the absence of any kind of explicit 
response requlrenentV Since studies of response mode/requi- 
rement mainly concern explicit response requirements,^ and^ 
since implicit response- requirement are so very elusive, if 
they at all exist, the further discussion of response re- 
quirements will primarily be about explicit response requi- 

^ 'fei^^nts. The term 'response requirement' will therefore, 
throughout this book^ mean 'explicit response requirement'. 

Overt response requ^irementwill 
consequently mean an explicit demand for an observable' act 
of responding. Covert response require- 
ment will mean an explicit demand for a non-observable 
(mental) act of responding. No response, re- 
quire m e n t,' finally, will mean the absence of an ex- 
plicit demand for a response, assuming also - unless other- 
wise stated - that there is no response blanks in the pro- 
gram nor any other space designated for the recording of 
student responses. 



2.4 ;PRONPTING 



The term 'prompting' is often used in the PI literature.. 
Usually it is a label for all kinds of features or cues in 
program frames, which have the objeptive of making the Re- 
mission of the correct answer more likely", t^_ use the 
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v/ords of Meyer (107). The degree of prompting can vary from 
mere hints to maximum or qpmpTete prompting, which means 
that the student is told what the response should be, before 
he is asked to respond.^ & - 

\ 'Prompting" is often "used to label the last-mentioned type 
of complete prompting (e.g. in 82:V34J. This is particularly 
common in studies of 'prompting versus confirmation' (e.g. 
69). In order to avoid vagueness and confusion, it is 
suggested here that the term 'cueing' (see for instance 67: 
13 & 82:134) should replace 'prompting' in its general sen- 
se, while the term 'prompting' -should 'only be used to de- 
note' complete cueing as described above. 

Like s^^raU^f the terms already discussed in this chapter, 
,.the term 'prompting' has the disadvantage of being ijable to 
imply ai^sumptions concerning student behaviour, which may be 
difficult to control. In a program sequence of the prompting 
type (key answer presented before the student responds) it 
is possible that v/hat is meant to be prompting, can in fact 
become confirmation, namely if the student responds - pro- 
bably covertly^- before the key answer is perceived (cf . 
71). This vagueness can, however, be- avoided, if 'pranpting' 
1s defined as the program property (key answer presentation 
before student responding) that;/is expected to have a promp- 
ting function, A recent discussion of prompting can be found 
in a paper by Richard Anderson (60), v^ho uses the term in a 
very general sense. 



2.5 RESULT VARIABLES ETC.' 



An evaluation of an instructional process should, in prin-. , 
ciple, take three main factors into account: 

- the increase in student performance or ability, re- 
sulting from the process under studyi 

- the student working time needed for the process; and 

- other costs of the process. 

The term 'effect' is often used to label the first of these 
three factors, the incrgase In performance, while the term 
'efficiency' is used in the case of simultaneous evaluation ' 
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pf the two fif-st factors, e.g. performance' increase per unit 
of student working time (cf. 101 & 118). 

In the present study the terms ^effect' and 'efficiency* 
will be used in the above-mentioned way. Thus, 1 e -a r - 
ning effect' will denote an increase in student 
performance, which is assumed to be a result of the instruc- 
tional process. Where there is no, risk of confusion only 
•learning' or only 'effect' will be used in the same sense. 
When one and the same test, or parallel tests, are used to 
'measure performance both before and after the Pip the measu- 
red effect /nay of course to some extent be due to a prac- 
tice effect from the first testing. This can. hov^ever, be 
taken into account in the definition of learning effect 
simply by considering the pretesting procedure as part^of 
the instructional process studied. It should also be ob- 
served that pretest effects do not necessarily occur in PI. 
Hartley (89) has studied this problem rather extensively 
and found no such effects. 

The term '1 e a r'n i n g efficiency'- or, where 
^here 'is no risk of confusion, only 'efficiency - v/tll in 
this study denote the learning effect in relation t(5 P I 
time. The term 'PI tirrte' denotes the gross time spent by 
the student on" the program, i.e. the sum of the study pe- 
riods devoted wholly to the program, The^temi learning 
time' is avoided primarily because this term should denote 
the net or effective working time, and of this v/e generally 
know very little, except that it can safely be assumed to be 
shorter than the PI time. 

Efficiency could be more strictly defined by the formula 

Efficiency = Effect/PI time. 
In practice it is, however, difficult to obtain reliable- ,^ 
measures of this nature. Efficiency must, therefore, often 
be estimated in other ways. More will be said about this 
problem in section 3.3.2.4 below.. 

As defined here efficiency does not include any reference 
to any aspect of the third of the factors listed above, the 
costs of the process, other than student time. This could, 
of course, easily be remedied by changing the definition so 
that 'efficiency' means 'effect per working time unit and 
cost unit' (cf. 21:117)/The difficulties of measuring effi- 
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clency defined in that vvay in the field of education would, 
hoi'^ever, be almost Insurmountable. 

In roany cases one tries to measure immediate learning effect 
as well, as retention some .time after the end of the instruc- 
tional process. The term learning effect' will be used in 
this study to denote 'immediate learning effect'. The effect 
remaining some time after the instruction has finished will 
be labelled, as it generally is, 'r e t e n t i o n' or 
Ivong-term learning effect'. 





•3 / 

Previous Research 



.3.1 OUTLINE OF THE REVIEW OF PREVIOyS*' RESEARCH 

. Research works ana scientific discussions^concerning re- 
sporiSrmode/requirement and use of key answers in PI using 
linear 'programs are reviewed here with an emphasis on the 
results of practical educational research rather than on 
/ learning research. A complete inventory of all that has^ 
been written about these topics has not been attempted. I _ 
have regarded it as sufficient to identify the major trends 
in the results of previous research. 

Several research reviews "vering the problems studied here 
have been published (4. 59. 61 . 70, ^1 .90' 98. 99. 116 & 
123^ Most of these reviews have been of the traditional 
•tvpe- Sat is they have consisted of short abstracts and 
Sent of a critical, evaluative or comparat ve character. 
The review of Bernmalm (4) differs from the others in that 
data concerning the investigations are Pi^^sented in tables, 
which facilitates their use for the study of Problems other 
tha^those explicitly treated by the reviewer. A thorough. 
Sret?Sl aSa ysis of different aspects ff response mode 
and so called feedback is found in Goldbeck & Briggs (84). 

Most of the studies I have come across in my literature re- ■ 
view have been discussed in one or several of the reviews 
listed above, and I will therefore not present here another 
review o?tS; traditional type. Instead, I will try and give 
a summary of a more objective-analytic kind. More than fO 
studies of response mode/requirement or key-answer frequency 
in PI have been classified according to different program 
and student variables, and according to the results of com- 
parisons between different experimental groups in terms of 
'learning effect and PI time. Those classifications have 
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been made on the basis of data presented In the reports re-- 
viewed. Summaries have been made\in the form of c^oss-tabil- 
Utions, where the uflit of observation is the- individual 
study. This procedure makes it possible to study the impor- 
tance of certain program and student variables for learning 
effect and PI time by means of statistical description. 

Hie value of such summaries of groups of experimental stu- 
dies is limited-by \the following circurDstances: 
, - Investigations with the same objective are not seldom 
so different in 'design as to diminish their comparabi- v 
lity. • ^ ^ ^ 

- Deficiencies in the reporting of data concerning experi- 
ments sometimes make classification difficult or even 
impossible. 

-As all studies are given equal weight in -the sumaries^ 
it is not possible to account for differences between 
studies' in terms of subject sample size, scientific 
quality etc. 

As regards the third factor listed above, it should be no- 
ted, though, that if two comparisons have yielded 'signifi- 
cant differences using samples of xlifferent sizes, the stu- 
dy using the s m a 1 ^-e,.r^ sample should be giVen the 
greater weight, since the probability of obtaining signifi- 
cant>esults is smaller the smaller the sample. This has 
been pointed out by Bakan (64). W^ien the sample is large it 
is even possible that a statistically significant differen- 
ce Is practfically insignificant. As Nunnally puts It: 

"..If the null hypothesis is not rejected, it is usually 
because the N is too small. If enough data are gathered 

, the hypothesis will generally be rejected." (113:643) 

Concerning scientific quality there- exists, as far as I 
know, no measuring device or procedure of demonstrated re- 
liability and validity. Nasatir & Elesh (112), for instance, 
have Revised ap instrument for that purpose, but analysing 
their own vaTidation data one finds that the correlation 
between raters is somewhat- low e r than could be expec- 
ted if rating scores were totally randomly distributed! 
Many other instruments have been proposed j a review by Bar- 
tos (65) lists more than 20 items* for the period 1962 - 
1968. However, these instruments do not seem to have been 
empirically investigated as to their reliability and 
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validity. 



An three types of deficiencies listed above seem due to 
lack of co-ordination of educational research and to in- 
sufficient standardisation of research methods, variable 
definitions, methods of presenting data. etc. Research re- 
views of the type presented here can therefore, in addition 
to fulfilling their main function, contribute to the demon- 
stration of the co-ordination and standardisation problems 
in educational research. The classifications of research 
results made here constitute compromises 'between what Is 
desirable and what is reasonable in view of the aya lab e 
data. Deficiencies and uncertainties in the classifications 
will be.;discussed in connection with the presentation of 
each classification schema below. . ' 



3.2 CLASSIFICATION OF EXPERIMENTS 



3.2.1 . Comparison results 

The results of comparisons between paired groups are- usu- 
ally rendered in the form of gfoup means and t^values.- or 
z-values=~- for mean differences. But some studies only re- 
Dort whether differences are significant or not, and wheth- 
er they are positive or negative. And somepmes the signs 
of nonsignificant differences are "Ot/^PO"^*^^: J" ^J^.f °! 
these circumstances the following scale seems the most rea 
sonable basis for-the classification of the results of 
group mean comparisons: 

++ = positive significant difference 

+ = positive nonsignificant oifference 

0 = no difference or nonsignificant difference of 

unknown sign 

- = negative nonsignificant difference 

— = negative significant difference. 

Concerning what is significant or nonsignificant the state- 
ments 'made in the reports studied have been accepted. If 
the result of significance testing is reported for more 



than one level of signtficance, the results at the 5 per 
cent level are chosen. 

Comparisons between groups as to learning effect in the pro 
per sense (cf, definition tn section 2,5 above) are rare in 
the studies reviewed here. As a rule comparisons are made 
1n terms of posttest scores vnthout correction for differen 
ces in initial performance. For want of more precise data i 
have chosen to accept pQsttest differences as estimates of 
the corresponding learning effect (differences, 

PI t1ma is one of the very few variables in PI studies whic 
is normally measured in a quotient scale,' namely number of 
minutes or hours: The results of comparisons between experi 
mental conditions can, therefore, be expressed as quotients 
the PI time for each experimental condition divided by the 
PI time for ivhat can be regarded as the basic condition. As 
basic condition in the computation of such a quotient is 
chosen the experimental condition which logically should 
lead to the shortest PI time. Such PI tisne quotients, .com- 
puted from. data reported for diffe^^nt experiments, can be 
considered directly comparable. 



3.2.2 Program length 



The most common, and usually the only, measure of program 
length tn the PI literature is number of program units, or 
frames. The length of a program unit can, of course, vary 
considerably betv/een programs - and also within one and the 
same program. Most research reports do not permit any reli- 
able estimate of the length or 'size' of frames in the pro- 
grams used. According to Yaeger (125) it seems very diffi- 
cult to find a useful measure of program frame length. 

In studies of the effects of different response requirements 
and key answ-sr frequencies, one of the most interesting a- 
spects of program length is the number of responses demandec 
in the prograpi. If one response per frame is required, the 
number of frames is a suitable measure, but in other cases 
this measure tan be misleading. Unfortunately, the number 
of responses demanded in a program is rarely reported in the 
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studies reviewed here. Therefore, the number of frames is 
the only generally available measure of jprogram length. 

Taking the observed variation in the number of frames per 
program in the studies reviewed into consideration, the 
follovring classification schema has been established for 
the Vfiiri able program length: 

Category .. Definition 

Long More than 200 frames 

I - Medium lt)0 - 199 frames 

Short Less than 100 frames 

Unknown Number of frames not reported 



3.2.3 Students' age or educational level 

Research reports usually indicate e j t h e r the age of 
the students o r their educational level ("college , se- 
condary modern", "grade 9", etc.). For the classification 
of experiments In this respect I have defined the follow- 
ing student categories: 

16 years or more, 

13-15 years, 

10-12 years 5 

9 years or less, ' ^.4. 

age not reported and impossible to estimate. \ 
Where only educational level is reported the students are 
considered to be 6f the age which is noi^al for the educa- 
tional l.evel in question. 



3.2.4 Learning task characteristics 

a . • 

A classification of experimental studies according to the 
degree of learning task difficulty would be of great inte- 
rest in. this context, and so would a classification accor- 
dinq to the occurrence of unfamiliar response terms (cf. b^, 
141) i.e. what is often labelled the degree of 'response. 



learning* (e.g. ,35:504). Since sufficiently detailed pro- 
gram descriptions are not found in most of the reports re- 
viewed, I >)ave not considered it possible to construct a 
schema for objective classification of experiments accord- 
ing to such learning task characteristics. The question of 
possible interactions between response requirement or key 
answer frequency, on one hand, and different learning task 
characteristics, on the other » will therefore have , to be 
discussed in more informal terms (see section 3.3.2.3. 
below). 



3.3 STUDIES OF RESPONSE REQUIREMENT 



In vievi/ of what has been said about the terminology in this 
field (see section 2*3 above)^, it seems natural to classify 
studies of response mode or response requirement in PI 
wholly on the basis of program properties. The following 
main types of response requirement can be distinguished: 

overt, student-constructed response required (OCR); 

overt, multiple-choice response required (OMR); 

covert, student-c?nstructed response required (CCR); 

coverts multiple-choice response required (CMR); and' 

no response required (NOR). » 

The four first-mentioned types (OCR, OMR, CCR & CMR) may 
occur with or without a key answer^ following the student 
response, and they may also be canbined with prompting or . 
cueing. In practice, prompting is,, however» unusual when 
multiple-choice' answers are required. 

For obvious reasons key Answers or other kinds of inform^-^ 
tion regarding the correct responses do not occur in connec"- 
tion v/ith no-response requirement (NOR). Cueing may, however, 
occur in the NOR condition, when certain terms or concepts' 
are emphasized (cf. section 2.3 above). This NOR condition 
comes quite close to the CCR condition combined with cueingo 
the only difference being that the CCR'^ condition presupposes 
an explicit demand for student responses. The NOR condition 
is often called "reading", a term which seems most adequate 
when cueing does not occur. 




In the literature reviewed here several experimental studies 
comparing two or more response requirements are reported. 
The number of comparisons for each pair of requirements is 
. given in t a b 1 e 3 . 1 . As this table shows, a few 
types of comparison dominate the literature, namely 

- OCR requirement (overt constructed response) versus OMR 
requirement (overt multiple-choice); 

- OCR requirement versus CCR requirement (covert construc- 
ted response); ' u 

- OCR requirement versus NOR requirement (no response re- 
quired)t and 

- CCR requirement versus NOR requirement. 



Table 3.2 Number of comparisons of pairs of response 
requirement in the literature reviewed. 





OCR 


OMR 


CCR 


CMR 


NOR 


OCR 




6 


23 


0 


24 


OMR 






1 ' 


2 


2 


CCR 








0 


8 


CMR 










0 


NOR 













Note: The total sum of matrix values exceeds the 
number of studies reviewed, since several 
studies involve more than one comparison. 



In the following survey only the four most frequent compari 
sons are discussed, i .e.. OCR-OMR. OCR-CCR, OCR-NOR, and 
CCR-NOR. 



3.3.1 Cons tructed versus multiple-choice response re - 
quirement 



The li'terature reviewed contains five reports (12, 17, 20, 
41, 53) of studies where , student-constructed response is 
compared to multiple-choice response. In the study by Fry 
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(20) the t\40 response requirements are compared under two 
different conditions *of PI time, namely variable time and 
fixed time, which gives in all six comparisons. In all the 
six cases overt^responses are required, i.e. the require- 
ments compared are those here labelled OCR and OMR. The key 
answer frequency was 100 per cent throughout, i.e. key an- 
swers ware presented to students after each frame. 

Soma data regarding the six comparisons are summarized in 
table 3 . 2 • As the table shows, the comparison re- 
sults in tertus of'learning effect are either neutral (3 
cases| or positive, favouring the constructed response (3 
cases). The positive differences are significant in tv^o ca- 
ses» but these are not fully independent, sines they com^e 
from experiments made by the same researcher, using the 
same program (20). Comparisons as to retention have been 
made in five cases, and the results agree, on the whole, 
with the results shown in table 3.2. 



Table 3.2 Data on studies comparing OCR mth OMR^^ 



Ref 


Effect 


Pi time 


Student Program Program title 




OCRrQMR OCR/OMR 


age 


length 


12 


0 


V.2 


>16 . 


104/56^ Elementary Psych. 


17 


+ 


1.1 


>16 


72 Symbolic Logic 


20 


++ 


1.7 


13-15 


— ^ Spanish Vocabulary 


20 


++ 


1^ 


13-15 




41 


0 




^16 


1152 Analysis of Be 


53 


0 


1.3 


>16 


192 haviour 



IJ "Small" and "large" steps respectively 

2) 16 items repeated "several" times 

3) Same time for both conditions, preset by the experi- 
mentator 

The 91 time is reported in four cases, and all four PI timp 
quotients , (OCR-time divided by GnR-time) are greater than 
1, i.e. the PI time increases when overt student-construc- 
ted responses are required instead of overt multiple-choice 
, responses. Apparently, .tfie. size of this increase in PI time 
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can vary considerably. In the studies reviewed here the PI 
time under the OCR condition is between 8 and 70 per cent 
longer than the PI time under the 'QMR contoion (see table 
3.2). 

Learning efficiency is difficult to estimate on the basis 
of the data in table 3.2. It is obvious, hov/ever, that a 
demand for overt, student-constructed responses may in- 
crease PI time with up to 27 per cent - compared to the PI 
time using multiple-choice questions - without any signifi- 
cant or even noticeable increase in learning effect (53).. 
There is, therefore, reason to suspect that OCR requirement 
can lead to lower learning efficiency than OMR requirement. 
In a complete evalQation of the relative merits of these 
two response requirements the fact that program production 
costs increase when multiple-choice items must be provided, 
should, however, also be te^ken into consideration. 

The outcome of comparisons between the OCR and the OMR con- 
dition as to learning effect may also be influenced by the 
nature of the learning task,vespecially by the degree of 
response learning called for^fn the program (cf. section 
3.2.1.5 above). When a program has the aim of increasing 
the student's response repertory, e.g. his active vocabula- 
ry, it seems likely that the demand for student-constructed 
responses should have a particularly good effect. It is, 
therefore, 'interesting to note that the significant positive 
effects of OCR as compared to OMR, reported in table 3.2, 
have been reached using a program with the objective of' 
teaching Spanish vocabulary (20). 

The following tentative conclusions can be drawn from this 
survey regarding the demand for student-constructed respon- 
ses as compared to the demand for multiple-choice re^Sponses: 

- Student-constructed responses lead to equal or better 
learning effect. 

- They may in some cases lead to lower learning efficiency. 

- They lead to higher effect the higher the degree of re- 
sponse learning called f^r in the program.- 
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3.3.2 Overt or covert or no response requtrenient U 

In the literature reviewed here more than 30 reports have 
been found, where two or more of the response requtrements 
OCR, CCR and NOR are compared as to their effect tn PI using ' 

o linear programs. One study reports 3 e)(periments (1), three 
report 2 experiments each (2, 21 & 35), ^nd 32 studies re- 
port 1 experiment each (6-8, 11, 13, 14, 16-18, 22-25, 29- 
31, 33, 36-38, 4Q, 43, 44, 46, 47, 49-55 & 60). Since multi- 

^ pie-choice responses are not at all discussed in this sec- 
tion, the terms 'overt' response' and 'covert resptonse' are 
used to denote 'overt student-constructed response^ and 
'covert student-constructed response' respectively. 

Overt versus covert response requirement is studied in 24 
experiments (1, 2/2 exp:s/, 6-8, 13, 14, 17^ 21/2 exp:s/, 
33, 35/2 exp;s/, 36-38,. 40, 46, 47, 50-52 & 60). In all the- 
se cases the response requirements have been studied in com- 
bination with 100 pj2r cent key answer frequency. 

Overt response versus no response requirement has been stu- 
died in 24 experiments (1/3 exp:s/,'6, 8, 11, 16, 18, 21/2 
exp:s/, 23-25, 29-31, 43, 44, 49/2 exp:s/, 50, 53-55). In 
these cases, too, the OCR requirement has been studied un- 
vder 100 per cent key answer frequency. 

Covert re.^^se versus no response requirement has been stu- 
died in 8 experiments only (1, 6, 8-/21/2 exf^:?/, 22, 31 & 
50). In one of these (22) the covert response requirement is 
studied onlj^in combination with 0 per cent key answer fre- 
quency, while in all, the other cases the key answer frequen- 
cy is 100 per cent. Studies comparing the effects of diffe- 
rent response requirements also under key answer frequencies 
other than 100 per cent, are discussed further ^in section 
3,4 below. 

The results of the comparisons as to PI time, learning ef- 
fect, and retention in the experiments reviewed here are 
summarized in table 3.3. Since pretest data are 
usually not available, difference^ in posttest results ^»re 
used as measures of learning effect differences. As the va- 
riance of performance increments is normally smaller than 
the .variance of the correspo>iding total performance measure. 
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this procedure will profaafaly lead to an underestimation - 
rather than an overestimation of the degrees of signifi- 
cance associated with the reported differences. 

As table 3,3 shows, PI time data are availaMe for at?out^ 
• half of tf^e experiments reviewed. The retention of learning 
can be estimated in just over half of the experiments, but 
the time elapsed between th^ instruction and the second 
.posttest -^intendeJ to measure retention - varies consider- 
ably, from 2 weeks to 6 months. The distribution of compa- 
rison results concerning retention must, therefore, be in- 
terpreted with particular 'caution, 

/ 

Table 2.3 Studies of requirements OCR, CCR and NOR: 

outcomes of comparisons as to PI time (PT), 
learning effect (El), and retention (E2). 



Compa- 
rison 


Vari- 
able 


++ 


No. 

+ 


of comparison outcomes 
0 - — No data 


OCR-CCR 


PT 


12 


0 


1 


0 


0 


n 






r 
0 


7 


5 


6 


1 


0 






3 


3 


6. 


2 


0 


10 


OCR-NOR 


•PT 


9 


1, 


1 


0 


0 


13 




El 




5 


6 


5 


1 


0 




E2 


4 


4 


2 


1 


2 


11 


CCR-NOR 


PT 


1 


1 


1 


1 


0 


4 




El 


0 


2 


2 


4 


0 


0 




E2 


0 


1 


1 


1 


2 


3 
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1) ++ or + means that the mean for the fir^t- 
mentioned requirement ha? been higher than 
that for the last-mentioned. 



From the sunranary in table 3.3 it can be concluded that overt 
response requirement, whether compared with covert or with 
no response* requirement, lead$ to increasing PI time, but 
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that this - often considerable. - increase In Pl^time far 
from always results in a noticeable increase in learning 
effect. From this follows that the relationship between 
overt response requirement and learning efficiency is by no 
means unequivocal . 

Regarding covert response, requirement, as compared with no 
response requirement, one may conclude, from the summary in 
table 3.3*. that a demand for covert responses as a rule has 
a rather small influence on PI time as v/ell as on learning 
effect. ^ . 

The perhaps most important xonclusion'that can be drawn from 
data presented in table* 3.3 is, however, that any 'dogmatic* 
opinion concerning the genferal value of on^ specific respon- 
se requirement seems untenable. 

Consequently, a more detailed analysis Is desirable. Accord- 
ing to Holland {90:93ff) factors such as program length and 
the dTfficulty and relevance of the program question^ should 
be expected to affect the result of covert responding in PI. 
Ot^r researchers have pointed to the characteristics of the 
learning task, especially its degree of ''response learning" 
or "assodarfive learning" {35'& 59), and the age or intel- 
lectual maturity of the students (35) as important factors. 
The exp'eriments revievied here can, with few exceptions, be 
classified according to program length as well as student 
age. 

An objective classification of program properties such as 
level of difficulty, relevance in relation to the criterion 
test, and the degree of response learning called for, is in 
most cases not possible on the basis of the available data. 
The relationship between such program properties on the one 
hand and learning effect on the other will, therefore, be 
analysed by means of a discussjon of selected experiments. 
It should be noted here that level of program difficulty 
and degree of response learning are not program variables 
in a strict sense (see section 2.1 above), since they are 
to some extent dependent on student characteristics: a re- 
sponse term unknown to A may be well known to B, etc. 



3,3.2.1 Response requirement and program length 



In table 3.4 program length is cross-tabulated 
against the outcomes of response requirement companspns as 
to learning effect. 

The result of this cross-tabulation supports the hypothesis 
about a correlation between program „length and effect of 
overt response requirement. In all experiment^? yielding a 
significant result favouring overt responses (++),* programs 
with more than 100 frames have been used. Wheji short pro- 
grams - with less than 100 frames - have beerj used, most 
comparisons favour other response requirements. 

Judging from table 3.4, the correlation between program 
lengtfi and outcome of comparisons is higher when overt re- 
'sponse requirement (OCR) is compared with no response re- 
quirement (NOR) than when it is compared with covert re- 
sponse requirement (CCR). For short programs, negative out- 
comes are more often associated with the comparison OCR-NOR 
than with OCR-CCR. 

Regarding the effect of OCR. compared to CCR, the-results of 
this analysis support Hollands opinion that the disadvantage 
of covert responding appears first when the program is 
"lona enough for subjects in the covert mode to become 
careless..." (90:93). 
Mt-should be noted, though, that overt responding, and the 
increased PI time which it entails, means that a larg^ num- 
ber of short pauses occur in the students' intake of infor- 
mation, viz. when responses are recorded. To the extent that 
such pauses have a positive influence on learning, it seems 
reasonable that this influence should be greater the longer 
the program. 

As regards CCR compared to NOR, the small variation between 
studies in program length (table 3.4) allows no conclusion 
concerning a correlation between program length and outcome 
of comparisons between these two response requirements. 

A cross-tabulation co^^responding to the one in table 3.4 has 
also been performed for the outcomes of comparisons be^vjeen 
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Table 3,4 Cross-tabulation: Program length x outcome of 
comparisons between pairs of response require- 
ments* as to learning effect. x 





r rog rain 


Kin 
(MO . 


of outcomes type 


oUm 


rison 


1 ength 


++ 

o 


+ 


U 








OCR-CCR 


5^200 


2 


1 


o 
0 


1 


0 


7 




100-199 


2 


1 


1 


1 


0 


5 




<100 


0 


3 


1 


3 


0 


7 




unknown 


1 


2 


1 • 


1 


0 


5 


OCR-NOR 


^200 


2 


0 


1 


0 


0 


3 




100-199 


4 


1 


2 


0 


0 


7 




<100 


1 


2 


3 


5 


1 


12 




unknown. 


0 


2 


0 


0 


0 


2 


CCR-NOR 


5*200 


0 


0 


0 


0 


0 


0 




100-199 


0 


0 


0 


2 


0 


2 




<100 


. 0 


2 


2 


2 


0 


6 




unknown 


0 


0 


0 


0 


0 


0 



Table 3.5 Cross-tabulation: Program length x outcome of 
Comparisons between pairs of response require- 
ments as to retention. 



Compa- 


Program 


No. 


of outcomes type 


Sum 


rison 


length 


++ 


+ 


0 








OCR-CCR 


^200 


1 


1 


1 


2 


0 


5 




100-199 


2 


0 


0 


0 


0 


2 




<100 


0 


1 


3 


1 


0 


5 




unknown 


0 


1 


1 


0 


0 


2 


OCR-NOR 




1 


0 


0 


1 


0 


2 




100-199 


3 


2 


1 


u 


0 


6 




<100 


0 


1 


1 


1 


2 


5 




unknown 


0 


2 


0 


0 


0 


2 



response requirements in terms of retention of learning. The 
result is shown in table 3.5. There is a close 
correspondence between the resiU> of comparisons as to re- 
tention and the results of comparisons as to learning effect 
{table 3.4). The correlation between program length and the 
effect of OCR appears also in the analysis of retention 
data. The hypothesis about the importance of program length 
for the effect of overt responding in PI is, pn the whole^ 
supported by the present research review. 

3.3.2.2 Response requirement and student age or maturity 
level 



The possibility of a correlation between overt response re- 
quirement (OCR), on the one hand, and student characteris- 
tics such as age, mental age, study skill, etc., on the 
other, has been discussed by Leith (35). He sees the overt 
response requirement as a means of maintaining the learner's 
attention and focusing it on the crucial elements of the 
learning task (35:504). To the extent that overt responding 
has such 4 supportive function there may well be less need 
for it the older or more mentally mature the students are. 
A definition as well as a thorough discussion of attention 
in the sense of th^ term implied here can be found in^a 
paper by Richard Anderson (60). 

The hypothesis that overt responding (plus knowledge of re- 
sults) has the effect of heightening the attention of the 
students has also been supported in a study by S1me & Boyce 
(119). This study does not concern PI in a proper sense, 
since Sime & Boyce measured the effect of questions versus 
statements concerning concepts taught in a tape-recorded lec- 
ture. As they used multiple-choice questions the tv^o response 
requirements compared correspond to the OMC (overt multiple- 
choice) and. NOR (no response) conditions respectively. Their 
results indicate that OMC requirement heightens attention as 
compared to the NOR requirement. 

Holland also seems to believe that overt responding primari- 
ly is a support for the student, and that, correspondingly,^ 
covert responding makes greater demands upon the study disci- 
pline of the student (90:93), If this is true, it is also 



reasonable to assume that overt response requirement is more 
Important the younger the st^udents are. Holland himself does^ 
however, not take, this possibility into consideration, but 
contends ^that overt responding should be required at all 
student age levels. 

Rippsy reviev/ecl 14 studies comparing overt and covert re-. 

sponse requirement in PI or other learning tasks (116). He 

concluded that ' 

"a number of .. studies indicated that overt responding 
produced greater achievement for students viho v/ere very 
young .. and who had lower Intelligence quotients" 
(116:218).- 

' Rippey also concluded that the superiority of overt respon- 
ding tapars off betiveen grades 4 and 6, and that it is 
little evident in grade 6. 

If there is a relationship betiveen student age and effect 
of OCRp this should shov/ in a cross\tabulation' of student 
age and outcomes of response requi reinvent comparison^. The 
results of such a cross-tabulation arfe shov/n in t'a b 1 e 
3 . 6 which J however, shows no ^:leart;ut correlation ten- 
dency. But the tabU demonstrates clearly that the distri- 
bution of experiments on the different age categories is 
skewed: most studies have used older students as subjects, 
particularly studies comparing OCR and NOR. 

It should fee noted, though, that 6 oul of 8 studies of OCR 
involving .students under 13 years of age have yielded lear- 
ning ^effects results favouring this requirement. The data 
sUTOarized here give, therefore, some support to the hypo- 
thesis that overt responding in PI is especially benefici- 
ent for the lov^er age groups. The lack of a more clearcut 
correlation in the data at hand may be due to the scarcity 
of experin^nts involving younger students. The analysis re- 
sults reported here also indicate, then, what kind of expe- 
, riments are most needed in the further studying of this 
problen. 
i> 

Soma of the experiments reviev^ed nere have been designed 
with the specific aim of studying the relationship between 
student age, n^cntal maturity, etc., on one hand, and the 
effect of overt response requirement, on the other. 




TabU 3,6 Cross-tabulation: Student age x outcome of^ 
comparisons between pairs of response requi 
rements as to learning effect* 



Comp.a- Student No* of outcomes type Sum 
rison- age + 0 - — • 



OCR-NOR 


>16 


2 


4 


1 


3 


1 


11 




13-15 


1 


1 


3 


4 


1 


10 




10-12 


3 


1. 


1 


1 


0 


6 




<10 


0 


0 


0 


0 


0 


0 


OCR-NOR 


^16 


6 


3 


5 


4 


0 


18 




13-15 


1 


0 


1 


1 


1 


4 




10-12 


1 


1 


0 


0 


0 


2 




<10 


0 


1 


0 


0 


0 


1 


CCR-NOR 


>15 


0 


1 


0 


2 


0 


3 




13-15 


0 


1 


1 


1 


0 


3 




10-12 


0 


0 


1 


1 


0 


2 




<:10 


0 


0 


0 


0 


0 


0 
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Leith & Hope (38) report significantly better learning eff- 
ect with OCR than with CCR when' the students* mental age is 
below 11 years. For students of the same chronological age 
vnth mental age above 11 years the two response requirements 
led to approximately equivalent learning effects. According 
to Leith (35), Biran, Clarke & Leith (5) found significantly 
better learning effect with OCR than with CCR for "low abi- 
lity" students in a secondary modern school, while the more 
"bright" students in secondary modern and junior school 
learned about as much under both response requirements* 

Leith a Ghuman (37) compared OCR with CCR as to learning 
effect at three grade levels; grades 3, 4 and 5 in granmiar 
school, all subjects being girls. The interaction between 
response requirement and grade level was not significant, 
but mean differences pointed to a decreasing effect of overt 
responding vdth increasing student age. In grade 5 of the 
grammar school the difference in learning effect betv/een the 
ti^ response requirements was evert significantly negative, 
i.e. at this age level (about 16 years) the learning effect 
of the program on Co-ordinate Geometry was definitely lower 
v^nen overt responding was r^equired than when covert respon- 
ding was required. 

In a recent study Lewis & Whitwell (96) compared the lear- 
ning effect of OCR and CCR for 11-year-olds apd 13-year-old^' 
both groups including boys of high to medium mental ability. 
Apparently, they found no significant interaction of age and 
response requirement, but mean values reported indicate that 
the superiority of CCR - covert response requirement - was 
more marked for the ll-year-olds than for the 13-year-olds. 
Thus, Lewis & WhitvjelTs study does not suppart the hypothe- 
sis of increasing effect of overt responding with decreasing 
student age. Data on high-ability and low-ability students 
of n years are, however, in better agreement; with this 
hypothesis, since they indicate that the negative effect of 
OCR occurs mainly among the-'high-ability students. For me- 
dium ability students there was no difference betv^een re-, 
sponse requirements as to learning effect. 

Lambert, Miller & Wiley (33) divided thp grade nine students 
m their study of OCR and CCR into three intelligence groups- 
high (IQ 12Q-162),- normal (112-119) , aVid low (76-111). They 
used a fairly long program with 843 frames, but report a 
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significant, positive effect of overt response requirement 
(OCR) only for students in the high IQ group. Their 
results runs, then, counter to the hypothesis that overt 
responding has its greatest positive effects at- low levels 
of intelligencfe. 

In an unforj^unately very incompletely reported study Aust- 
wick (63) found no difference in learning effect between 
OCR and NOft requirements when subjects were graduate stu- 
dents, biit when he replicated the experiment with If -year- 
olds in grammar school OCR led to better learning than NOR. 
Austwick concludes that 

"the guided instruction provided by the constructed 

response programme was aiding the learning process. 

for the younger students" (63:198) • 

A couple of studies by Entwisle, Huggins and Phelps (16) 
indicate that OCR may be most beneficient - or least detri- 
mantai: - for high ability students. Their low ability stu- 
dents did better under a CCR condition. The programs used 
were apparently very difficult, so the low ability students 
may have been handicapped by poor understanding of the ques- 
tions. Attempts to respond to questions that are too diffi- 
cult may be less efficient than being told the answers right 
away. 

Only half of the studies reviewed in the last paragraphs 
lend support to the hypothesis discussed here regarding a 
relationship between student age or matyHty level, on one 
hand, and the effect of OCR, on the other. Also, three of 
the studies supporting the hypothesis have been performed 
v/i^hin the same iltstitution , by the same team of research- 
ers. They should, therefore, not be considered strictly in- 
dependent pieces of evidence (cf. RosenthaVs view that 
"ten experiments performed in a single laboratory may be 
worth less than the same ten experiments conducted in diffe- 
rent laboratories", 117:153). 

A study by Lockard & Sidowski (97) deserves mentioning in 
this context. Although it is not a PI study, it illustrates 
the problem discussed here. In this study the effects of OCR 
and CCR on the learning of nonsense syllables are compared. 
Subjects were students in grades 4 and 6. For subjects from 
grade 4 the overt response requirement (OCR) ted to higher 
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effect, although not significantly so» while for grade 6 
subjects the OCR led to significantly lovier learning effect 
than the CCR. 

In a study comparing conventional programmed instrt)ict1on 
and students' reading programs while making notes at their 
own discretion, Poppleton & Austwick (114) found tha^t PI 
was more efficient with younger students (13 yearsj, while 
reading plus note-making was more efficient with graduate 
students. 

Summarizing the present analysis one may conclude ';hat the 
hypothesis, launched primarily by Leith, that tfie jrasitive 
effect of overt responding decreases with increasing stu- 
dent age or maturity, 1$ reasonable although not unchallen- 
ged. It is probable that age as such is of no consequence ^ 
but rather some characteristics that covaPy'-with,;^i|e: 'work 
discipline', study skills, background knowledge, 'etc. (cf. 
section 3.3.2.3 belovi). If that is the case« one Should ex- 
pect to find an interaction between student age /nd type of 
program as regards the effect of different response require- 
ments. A response requirement that is suitablj^for a certain 
type of program and a certain age group may,/ for instance, 
be unsuitable for another program taken by xhe same age 
group. This could in turn partly e)^plain the absence of a 
correlation between student age and outcome of response re- 
quirement comparisons in table 3.6. " 

It should be noted that the problem discussed here is part 
of the general problegi of ATI, i.e. aptitude-treatment In-*^ 
teractions, which has been much discussed in the recent li- 
terature. A quite extensive summary has been published by 
Bracht in the 1970 issue of the Review of Educational Re- 
search. The ATI discussed here need not be linear, i.e. the 
efficiency of a treatment need- not increase or decrease mo- 
notonously with some student characteristic. On the contra- 
ry, the picture provided by the studies reviewed here indi- 
cates that there may be some optimal relationship between 
student ability and program difficulty, where overt con- 
structed responding is particularly beneficient. 
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3.3.2.3 Response requirement and learning task characte- 
ristics 

"..explicit activity is likely to contribute to the phase 
of learning referred to as response learning (35:504; 
note that "explicit" is used here as synonymous to 
"overt"). ^ , 

"Clearly, an overt, constructed response should be re- 
quired from a student, if he is expected to be able to 
emit an unfamiliar, technical term" (59:141). 

As these two quotations show, both Leith (35) and Anderson 
(59) assume that there is a relationship between the effect 
of overt response requirement, on the one hand, and the de- 
gree of response learning called for in the program, on the 
other. A similar point, although not specifically "Mentio- 
ning response learning, was jnade by Reid already m 1964' 
(115:159f), It has, however, not been possible to classify 
the studies reviewed here objectively as to the nature of 
the learning tasks involved (cf. section 3.2.4 above). Also, 
the degree of response learning called for m a program can- 
not be estimated independently of student characteristics, 
since response terms unknown to one student may be well 
knovfli to an other.. 

The nature of the criterion test is also important in this 
context. The learning effect under overt response require- 
ment can be expected to be correlated to the. amount of pre- 
viously unknown response terms only if the learning of these 
terms is measured by the criterion test. As Holland has 
pointed out (90), student responding in PI will influence 
learning only if the responses required^ are ,i^elevant, i.e. 
relevant with regard to the criterion test. Holland ^and 
Kemp (91 & 94) also have pointed out that responses required 
in a program must be contingent upon the program content. 
Othervnse the student^s responding will have no relationship 
to his learning of the program content. Holland and Kemp 
have devised a special technique and a measure, the black- 
out ratio, for the extent to which responses required in a 
program arS contingent upon program content. 

Most stuai.es reviewed here" are not reported in such a way as 
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to pennit judgmants concerning the criterion test relevanc© 
of th6 required responses. The possible relationship bet- 
v/een type of program and effect of overt response require- 
ment mjay nevertheless be illustrated by means of a, summary 
of the' available data. 'Table 3.7 contains a sum- 
mary of some program and student data for those experiments 
which report significantly better learning effect, for at 
least ^ome student category, under the OCR than under the 
CCR or NOR conditions. 

J 

As table 3.7 shows, 12 different programs have been used in 
the 16 experiments sumarized. Four of these programs led 
to positive effect of OCR for soma student subgroups only. 
Three of these programs (5p 33 & 38) have already been dis- 
cussed (section 3.2.2.2 above). The fourth* a program en- 
titled "Electronics", was used in two studies (7 & 36) and 
' in both* cases the OCR led to significantly better learning 
than the CCR for students with "low background knowledge" 
' in physics, i.e. students for whom the program probably con- 
tained a large number of unfamiliar response terms. 

For eight different programs, used in eleven experiments, 
the OCR had a significant positiv:e effect for the whole stu- 
dent population. One of these program teaches spelling (35), 
a task very much characterized by response learning and, 
furthermore, a task where the viriting down of hesponses is 
particularly important, since what is to be learnt is the 
correct v/riting of words. Another program addresses itself 
to medical students, giving "elementary information" (14: 
238), which probably means that the students had little ini- 
tial knowledge of the learning material. For yet another me- 
dical program of "technical, unfamiliar" nature, Tobias & 
Abramson (49) found a significant difference favouring the 
OCR condition. 

Three of the programs summarized in table 3.7 treat psycho- 
logical or educational subject matter. This program "Funda- 
mentals of educational test interpretation", used in two 
studies by Krumbolz & Weisman (31 & 32), is said to have the 
objective of giving the students 

"a conceptual understanding .. of percentiles, age and 
grade scores, normal distribution curves, standard de- 
viations and z-scores" (32:90). 




Table 3.7 Program and student characteristics of experi- 
ments yielding significantly better learning ef- 
fect under overt (OCR) than onder covert (CCR) or 
no (NOR) response requirement. 



Program 

(title or subject) 



Student 
category 


No. 

of 
exp 


Ref. 


Com- 
pari- 
son 


Coll. stud:s 


2 


30.31 


Both 


II _ 

Medical stud:s 


3 


24.25,53 

43 

14 


NOR 

n 

CCR 



Fundamentals of test 
interpretation / 
Analysis of BehaVior 
Programmed instruction 
Diagnosis of myocardial 
infarction 
Medical terminology 
Spelling 

Definitions of marrDnals 
Vocabulary: difficult 
Electronics 

Molecular interpreta- 
tion of heat transfer 
Latitude & longitude 

Sets, relations & 
functions 



Boys, 
Grade 
Grade 
Coll 



49 

35 

51 
II 

7.36 



sec. mod. 1 
6 stud:s 1 
5 stud:s 1 
stud:s V/. 2 
low backgr.knowl . 
Sec.mod.stud:s 1 5 
of low ability 
11-year-olds, MA 1 38 
below IVyears 
Grade 9 stud:s 1 33 
w. high IQ 



NOR 
CCR 
NOR 
CCR 
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This probably Implies the use of a number of "unfamiliar 
and technical" response^ terms. About the program "Programed 
Instruction", used by Ripple (43), there is no information 
available in the research report. It does not seem unreason- 
able, however, to assume that a program with this title ♦con- 
tains a number of response terms, unfamiliar to the stu- 
dents; 

The program "The Analysis of Behavior" was constructed by 
Skinner and Holland, two of the most ardent supporters of 
the need- to require overt responses in PI. This program was 
used 1n three studies^ two by Holland himself (24 & 25) and 
one by Williams (53). It is worth noticing that this Skinner 
-Holland program, as weU as the program "Programed Instruc- 
tion" -mentioned above, was used only in comparisons between 
overt (OCR) and no (NOR) response ^requirement. Strictly 
speaking, this only demonstrates that explicit re- 
sponse requirements have a positive effect on learnirfgs and 
not that the overtness of the response activity 
has a positive effect. An explicit response requirement, 
: whether overt or covert, has the function of informing the 
student about what elements of the learning task - the pro- 
gram - should primarily be learnt. Such guidance may have a 
particularly great influence when the program is long; in 
a short program it may be possible. to learn all there^s to 
learn. 

The observations reported here indicate clearly a correla- . 
tton between the nature of the learjiing task and the effect 
of overt response requirement: the more unfamiliar response 
terms a program contains 9 the higher the positive effect of 
overt response requirement should be expected to be. 

This conclusion is also supported by the results of other 
kinds of learning experiments. Eigen & Margulies (80), for 
instance, compared OCR and CCR when the learning material 
consisted of nonsense syllables at d'ffferent "information 
levels". High information level in this case means that the 
association value of the syllable is low, i.e. the syllable 
is not similar to any existing word. The authors found that 
the positive effect of overt response requiremeht (OCR) in- 
creased with increasing information level , i.e.- when re- 
sponse terms became more "unfamiliar" to the subjects. At 
the same time, the OCR had, hov;ever, the effect of decreas- 
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ing the .amount of "incidental learning", i.e. learning of 
syllables not designated as response terms. , 

It is quite probable, however, that the relationship bet- 
ween the nature of the learning task and the effect of overt 
response requirement is much more complicated than it has so 
far appeared to be. Increasing learning task difficulty 
should be expected to leadHo an increase in the positive 
effect of overt response requirement. But if the degree of 
•difficulty is very high, this positive effect may fail to 
appear. A study by Goldbeck & Campbell (21) lends some sup- 
port to this assumption. In this study the authors used 35 
items df "factual learning". The degree of item difficulty^ 
was varied by means of cueing. The OCR condition had a nega- 
tive effect when items were easy, and a positive effect v^hen 
they were moderately difficult. But when items were very 
difficult, the difference between response requirements -as 
to learning effect was very small. The mean frequency of 
correct answers to the difficult items was only 21 per cent, 
a fact which caused Holland to remark: 

"If items cannot be answered anyway > it should make 

little difference wljether the failure to answer is 

in writing or in thought." (90:96) ' 
I agree, on the whole, with Holland here, but it should be 
pointed out that it may make a difference whether the 'fai- 
lure to answer is in writing or in thought". If by failure 
one means making a wrong response, .the writing down of the 
response in^^tead of merely thinking it may have a negative • 
effect on learning. 

The idea that overt responding may have a. negative effect 
on learning has also been put forward by Anderson (60) . In a 
balanced discussion of the problem of active responding he 
first says that 

"when a student can respond correctly to the frames 
without paying attention to critical material, atten- 
tion to this material is minimized and learning suffers. 
However, the requirement to make overt responses might 
be facilitative provided that correct responding was 
contingent upon attention to all of the critical mate- 
rial" (60:355). 

After a review of the Kemp & Holland study of "black-out ra- 
tio" (94), demonstrating that only cue-contingent overt re- 
sponses facilitate learning, Anderson notes: 
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"It may be true» hov/ever, that the requirement to make 
overt responses which are not cue-contingent disrupts 
normally adaptive reading habits." (op.clt.) \ 
It must be observed, though, that Anderson discusses \ 
active responding, i.e. overt responding as opposed \ 
to no responding, and that he does not take into considera-\ 
tion the possibility of active c, o. v e r t respon- \ 
ding. ' 

Suming up;: this discussion of the relationship between the 
nature of the learning task and the relative merit of qvert, 
covert or no response requirement, one may safely conclude 
that the research reviewed indicates that no predictions re- 
garding the effect of overt responding should be made with- 
out taking the characteristics of the learning task into* 
consideration. ^ 



3.3.2.4 PI time and learning efficiency 



Although different researchers hold different opinions about 
the value of overt responding in PI, there seems to' be .gene- 
ral agreement on one point: overt response requirement leads' 
ta longer PI tiiiie than covert or no response requirement. 
This is only natural, since the student, having decided what 
response to give, must dfevote some time to the recording of 
his response, that is. If he i^esponds overtly as required. 
If not, the overt response requirement will not lead to 
longer PI time than any other response requirement. 

In order to evaluate the influence of the response require- 
ment upon learning efficiency one must know both the learn- 
ing effect and -the PI time. Data regarding PI time are, how- 
ever, reported only in about 50 per cent of the studies re- 
viewed here (see table 3.3 above). 

For studies reporting ?l time data PI time quotients (,have 
been computed (see section 3.2.1.2 above), using the 1^1 time 
under the NOR or, v/here this is not included, the CCR condi- 
tion as denominator of the quotient. The resulting PIT quo- 
tients are given in table 3.8. From data in this 
table the follov/ing PIT quotient values can be computed; 
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OCR/CCR: 

Mean =1.67 ■ 

Variation width: 1.36 - 2.40 

OCR/NOR: 
Mean =1.78 

Variation width: 1.29 - 2.32 



The mean increase' in PI time following OCR is. then, 67 per 
cent of the PI time under the CCR, and 78 per -cent of the 
PI time undef the NOR. The lowest PIT quotient reported in 
any of .the 13 studies^in table 3.8 is 1.29. i.e. in no case 
did the OCR lead to an increase in. PI time of less than d9 
per cent, which is a considerable increase n the total 
working time spent on a program. Consequently, the OCR con- 
dition must lead to a considerable increase in learning ef- 
fect, in order to reach the same learning efficiency as the 
CCR or NOR conditions. 

Data available in the reports feviewed here give,' as a rule, 
small possibilities of computing measures of learning effi- 
ciency. This is mainly due to the fact that the absolute 
magnitude of the learning effect usually cannot be estima- 
ted learning efficiency is very difficult to measure under 
any circumstances. Firstly, this measuring presupposes that 
learning is measured in an approximate qliotient scale, as 
has been pointed out by Holland (90:102 . Secondly, compan 
ffsoSs betSeeS quotients of learning effect tjroug learn ng 
time oresuppose a linear correlation between time and learn 
ing. This condition Is not likely to be met in practice, 
since curves relating learning time to performance are usu- 
negaJfve'Iy accelerated, i.e. a given increment produces 
a smaller increment in performance the longer the total 
learning time. 

Holland's criticism of the habit of computing "efficiency 
index by dividing test score by learning time (90.102) is 
fully justified, but the efficiency problem cannot i?e dis- 
missed, as Holland seems inclined to do, by such criticism. 
This point was also made by Lumsdaine, who put forward cri- 
tical viewpoints similar to those of Holland and ended. 
"These considerations .. do not controvert the 

need to take time into account 

.." (101:310; my italics) . 



Table Z.d PI time (PIT), expressed as PIT quotients, 
under different response requirements. 



Refe- Type of PIT quotient 
rence OCR/CCR OCR/NOR CCR/NOR 



14 


1.90 






17 


1.61 






21 




1.20 


0.95 


21 




1.80 


1.21 


25 




2.14 




'30 




2.32 




38 


1.66 






44 




1.5Q 




47 


1.38 






49 


2.40 






52 


1.47 






53 




1.65 




55 


1.37 







Out of the 13* experiments summarized in table 3.8 only one 
contains data on the level of learning reached by students 
not taking any version of the program, namely the study by 
Williams (53). This study reports a posttest mean of 8.8 
for a control group not taking the program. If this value 
is substracted from the experimental group means, one gets 

Mearning effect* under OCR = 14.7; and 

'learning effect' under CCR = 11 ..S, 

Assuming a quotient scale,^ the increase in learning effect 
follovring the OCR would be 25 per cent of the effect under 
the CCR. But the corresponding increase in PI time is 65 
per cent, according to Williams's data. Admittedly, this 
analysis is based on some unwarranted assumptions, but it 
shov^s nevertheless that, although the learning effect 
was highest under the OCR, it is quite possible that the 
learning efficiency was appreciably lower under 
this requirement than under the CCR. 



If quotient scales for the measuring of learning efficiency 
are not available, one may attempt to judge efficiency by 
means of a conclusion chart based on Boolean algebra. If the 
relationship between the means .of two experimental groups 
(1 and 2) in the variables effect (E) and PI time (PIT) can' 
be expressed by < , = or > , one can drav/ one out of four 
" different conclusions regarding the relationship between the 
tm groups in terms of efficiency (EFF): 



(1) EFF(l) > EFF(2) if 

- E(2) cmd PIT(l) < PIT(2)) or 
E(2 on^ PIT 1) < PIT(2))s 



m : 



(-2) EFF(l) = EFF(2) if 

^ E(l) = E(2) and PIT(l) = PIT(2); 

(3) EFF(l) < Eff(2) if 

^ (E(l) = E(2) and PIT(l) > PIT(2)) or 
(E(li < E(2) and PIT(l) > PITC2)); 

(4) EFF(l) ? EFF(2) /i.e. the relationship is 
undetermined/ if 

(E(l) > E(2) and PITfl) > PITf2)) or 

(E(l) < E(2) and PIT(2) < PIT(2)). 



This conclusion chart involves no comparison as to magnitu- 
de betvfeen a difference in E and a difference m PIT. In re- 
turn one must allow for the possibility of an undetermined 
relationship in terms of EFF. i.e. vjhere it is impossible 
to judge whether EFF(11 is greater than, equal to. or smal- 
ler than EFF(2). This is expressed in conclusion no. 4 ar 
bove. 

If the relationships in terms of E and PIT are defined as 
follows: 

> : significantly greater than; 
"= : not significantly different from; and 
< : significantly smaller than; . 
then one has a crude but usable schema for conclusions re- 
garding the relative efficiency of diggerent experimental 
conditions. 

Even with this rather simple procedure one runs into diffi- 
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culties vrtien one tries to apply it to data available in the 
studies reviev/ed here* Firstly, PI time data are needed, 
but such data are reported only in about half of the studies 
(see section 3.3.E above). Secondly, measures of learning 
effect (£) are needed, and such measures are, as we have 
seen, rare (section 3.2.1.1 above). For v^ant of better data 
one must, then, assume that the relationship between group 
posttest means reflects the relationship in terms of effect. 

The schema for conclusions regarding relative learning effi- 
ciency proposed here is applicable in 12 cases of comparison 
between overt (OCR) and covert (CCR) response requirement, 
and in 10 cases of comparison between overt and no (NOR) re- 
sponse requirement. The resulting conclusions are summarized 
in table 3.9. As this table shov/s, the conclusion 
is tn 9. cases out of 12 that OCR is less efficient than CCR, 
and in 6 cases out of 10 that OCR is less efficient than 
NOR, In all the remaining cases the conclusion is that the 
relationship is undetermined. It is possible, then, 4;hat 
the OCR has been less efficient than the alternative' respon- . 
se requirement, CCR or NOR, in all the 22 cases of com- 
parison reviewed here. 

Obviously, this evaluation of the relative learning effi- 
ciency of different response requirements gives no support 
whatsoever to the hypothesis that overt response require- 
ment 15 more efficient than covert or no response require- 
ment. On the contrary, one must expect efficiency to be 
lower when overt responses are required* 

To neglect registering and reporting, in studies of re- 
^pbnse requirement^ the time^^pent by students working on 
the program, diminshes in view of these findings, conside- 
rably the value of such studies, since it allows no estimate 
to be made of the learning efficiency of the different ex- 
perimental conditions. 



\ 

\ 

Tcile 3.9 Learning efficiency of different response 
requirements: Summary of conclusions accor- 
ding to a special schema. 



(a) OCR versus CCR 



Conclusion 


No. of 
exp:s 


References 


1: OCR more efficient than CCR 
2: OCR and CCR equally efficient 
3: OCR less efficient than CCR 
4: Relationship undetermined 


0 
0 
8 
3 


2^^ ,17,2f\33,47,52 
14, 38,60 


(h) OCR verauB WR 


Conclusion 


No. of 
exp:s 


References 


1: OCR more efficient than NOR 
2: OCR and NOR equally efficient 
3: OCR less efficient than NOR 
4: Relationship undetermined 


0 
0 
6 
4 


2f'^,23, 44,49, 55 
25,31*49,53 



2 experiments 



3.4 INTERACTION OF RESPONSE REQUIREMENT AND KEY ANSWER 

FREQUENCY 



All comparisons of response requirements reviewed so far 
have been made in combination with 100 per cent key answer 
frequency for overt and covert response requirement, and, of 
course, 0 per cent key answers for no response requirement. 
It is possible, however, that there is an interaction bet- 
ween response requirement and key answer frequency as to the 
learning effect produced. 

Empirical studies 53f such an interaction in PI have been 
rare: only three instances have been found in my literature 
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revtev/. And bne of these» a study. by Fiks (19) » Is of a very 
special nature, in that 1t Involved visitors to an exhibi- 
tion vjho volunteered to take short - 20-24 frames - programs 
under different conditions: OCRp CCR or NOR, and 100, 50 or 
0 per cent key answer frequency. Fiks found no noticeable 
interaction betv/een response requirements and key answer 
frequencies as to posttest results, although he did find a 
significant interaction between these two factors in terms 
of attitude to PI. This attitude interaction implied that, 
under the OCR condition, subjects getting 100 per cent key 
answers had more positive attitudes to PI than those get- 
ting 50 or 0 per cent, t/hile there was no such attitude 
difference under the CCR or NOR conditions. 

Moore a Smith (41) compared OCR with OMR (overt, multiole- 
choice response requirement) under different key answer con- 
ditions: 100 per cent kev ansv/ers plus cash rewaf.d; and 0 
per cent key answers ancfno reward. The analysis of learning 
effect data yielded ho significant interaction between re- 
sponse requirement and response infomation. 

In a recent study Lewis & Whitwell (96) reported an experi- 
ment with an J in this contexts very Interesting multifacto- 
rial design. This design included, among other factors, a- 
bility (higho medium), age (11 or 13 years), "reinforce- 
ment" (ue. key ansv^r frequency: 100, 33 or 0 per cent), 
and "response" (overt or covert constructed response re- 
quirsfnent). Unfortunately, their data are somewhat unsatis- 
factorily reported - and data on PI time are also missing. 
Lewis Ei Hhitwell claim to have found some intriguing in- 
teraction betv/aen "response" and "reinforcement", but v/heth- 
er one judges by their retention test data alone or by sum 
scores for retention, "transfer" and "generalisation" tests, 
there is no clear interaction between response requiranent 
and key answer frequency in their data, 

A study by Michael fk Maccoby (108) deserves mentioning in 
this context, although it does not deal with PI. These au- 
thors studied the effect of questions in connection with the 
showing of instructional films; comparing OCR and CCR re- 
quirement and 100 and 0 per cent key answer frequency in a 
2x2 design. They failed, however, to find any significant 
interaction between response requirement and key answer fre- 
quency in their analysis of learning effect data. 



The results of this review of research in the interaction 
of response requirement and key answer frequency are, in- 
deed, meagre - too meagre, perhaps, to warrant even a tenta- 
tive conclusion. The studies reviewed suggest, however, that 
one should not expect to find any simple and clearcut rela- 
tionship between key answer frequency and response require- 
ment as regards their interaction in influencing learning 
effect in PI. 



3.5 STUDIES OF KEY ANSWER FREQUENCY 



The literature reviewed contains reports of 17 experiments 
concerning the effect of' varying key answer frequencies (ICF) 
in PI (3, 9, 10, 15 /2 exp/, 18, 19, 24, 30, 32, 34, 39, 41 , 
42, 43, 48, 49). In two-cases (32 & 39) 100 per cent KF is 
compared with 0 per cent KF as v/ell as with KF:s between 0 
and 100 per cent, i.e. intermittent response information. In 
four experiments (3, 15 /2exp/ & 30) 100 per cent KF is com- 
pared only v/ith intermittent response ioformation. In the 
remaining 12 experiments only the two conditions 100 per 
cent KF and 0 per cent KF are compared, 



3.5.1 Total elimination of key answers 



The outcomes of the 16 comparisons between 100 per cent and 
0 per cent KF as regards learning effect are distributed as 
follows (where positive sign means better effect vdth 100 per 
cent KF): 

Outcome: +^ + q - — 



No, of studies 0 8 4 3 1 

Only one study resulted in a significant difference, and this 
was a negative one, i.e. the learning effect was better with 
no key answers than v;ith 100 per cent KF (39). The distribu- 
tion of experiment outcomes lends no definite support to any 
hypothesis concerning the effect of key answers in PI - ex- 
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cept, perhaps, to the hypothesis that this effect Is gene- 
rally small and Insignificant. 



Soma cross-tabulations have been performed 1n order to Il- 
lustrate the possible Interactions of key ansi^r frequency 
and different program or student variables. la t a^b 1 e 
3 . 10 is shown the result of a cross-tabulation of pro- 
gram length, on ^e one handt and outcome of comparisons as 
to learning effect, on the other. (For definitions of the 
bases of classificaition see section 3.2-.1 above.) The 



come of comparison between 100 and 0 per cent KFr'seven out 
of eight positive outcomes involve programs vdth less than 
200 frames, whereas three out of four negative outcomes in- 
volv^ longer programs (more than 200 frames). In the study 
yielding a significant negative outcome of the comparison 
between. 100 and 0 per cent KF, the program used contained 
more than 1 ,000 frames (39).^ \* 

This analysis iQdicates, then, .that key answers, presented 
after the student has .responded, may have some positive ef- 
fect* fn shorter prograqts, but no or ev6n negative effect 1n 
longer programs. This seems far from unreasonable. That key 
answers, which are properly used for the purpose of stu- 
dents* response control, may have a positive effect on 
learning seems obvious (see e.g'. Anderson's discussion in 

.59:151). On the other hand, when students know that a key 
answer will alv/ays be provided imme^diately after the stu- 
dent's own responding, this may affect their way of.working 
with the program. That is how Lublin explains her finding of 
a negative effect of 100 per cent KF: 

"The task .. may have been unchal longing ..due to the 
100 % predictability that the right answer would im- 
mediately be supplied'* (39:299). 
In this situation the student may make less effort to find 

■ the right answer himself, which in, its turn should be expec- 
ted to affect learning negatively. If so, 1$ is also reason- 
able that this tendency should be-stronger the longer the 
program, since the temptation to get away with as little 
work as- possible probably increases with the amount of work 
to be dona, viz. when the program becomes longer. 




between program length and out- 
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Table Z.io Comparisons as to learning effect betv/een 
100 and 0 per cent KF: Cross-tabulation 
of program length and outcome of comparison* 



Program 


No. of 


comparison outcomes 


Sum 


length 


++ 


+ 


0 








>200 


0 


1 


1 


2 


1 


5 


100-199 


0 


4 


2 


1 


0 


7 


<100 


0 


3 


1 


0 


0 


4 


Sum 


0 


8 


4 


3 


1 


16 



T 



The argument just presented is also supported by the data 
on PI time reported by Lublin (39). She found that the mean 
PI time was longer for C^per cent KF than it was for 
100 per cent KF. If key answers are actually used for the 
purpose of response control t this should - other things be- 
ing equal - increase the total time spent on the program, 
i.e. PI time for 100 per cent KF should be longer than that 
for 0 per cent KF. Iff on the other hand, key answers are 
simply neglected, the PI time should be the same regardless 
of the key answer frequency. Lublin's results must therefore 
imply that the students in her study did, in fact, devote 
more time to the program frames, when they knew that 
key answers would n o t be providejl. 

As in the case of studies of response requirements in PI 
(see section 3.3.2.4 above), it is desirable that PI time 
data be registered and reported in studies of key answer 
frequencies, ^n most studies reviewed here this has not been 
done. It is therefore not possible to judge whether l^ublin's 
finding is typical, i.e. whether the elimination of key ans- 
wers In PI usually leads to an increase in PI time. 

Table 3. 11 contains a cross-tabulation of student age 
and outcome of comparisons- between 100 and 0 per cent KF as 
regards learning effect. There is no clear tendency of a 
correlation between the two variables: student age and the 
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effect of key answer frequency. It seems, therefore, rea- 
sonable to assume that hypotheses regarding the effects of 
key answers in PI can be formulated v/ithout regard to the 
age of the students involved. 



Table Z.ll Comparisons as to learning effect between 
100 and 0 per cent KF: Cross-tabulatipn of 
student age and outcome of comparisons. 



Student No. of comparison outcomes Sum 



age 




++ 


+ 


0 








^16 




0 


4 


3 


0 


1 


8 


13 - 


15 


0 


1 


1 


2 


0 


4 


10 - 


12 


0 


3 


0 


1 


0 


4 


<10 




0 


0 


0 


0 


0 


0 


Sum 




0 


8 


4. 


3 


1 


16 



Positive outcome means that the effect is 
better for 100 per cent KF. 



It has been suggested that the effect of knov/ledge of re- 
sults upon achievement in PI interacts with student ability, 
and Golloday (86) even contends that "numerous studies" sug- 
gest such an interaction. Still, Golloday himself cites only 
one study by Eigen (79) as example of those studies and this 
study contains, in fact, no real support for a hypothesis 
concerning such an interaction. Lewis & Whitwell {96> have, 
however, reported a significant interaction of "reinforce^ 
ment" (0, 33 or 100 per cent KF), "ability" {higt>, or medium), 
and "process" (type of posttest) for 11-year-olds. Their 
group means indicate that learning increases with 
increasing KF (from 0 to 33 to 100 per cent) for high abili- 
ty students, but that learning decreases with in- 
creasing KF for medium ability students. Lewis & Whitwell 
provide no explanation for this interaction, and it did not 
appear in their corresponding analysis of data for 13-year- 
olds. For the time being It seems wise to regard their 
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finding for the 11 -year-olds as one of these isolated, non- 
general isable research results that now and then plague us. 

The outcomes discussed so far in this section concern only 
the innnedi a teM earning effect. For a more thorough analysis 
of the effects of key OJiswers it would, of course, be de- 
sirable to have access to retention data. Such data are, 
hov^ever, available only in two of the studies reviewed here 
(43 & 96). Consequently, it has not been possible to ana- 
lyse the long-term effects of key answers in PI on the basis 
of the data at hand. 



3.5.2 Intermittent response information 



The distinction between partial reinforcement - as a learn- 
ing psychology concept - and intermittent response informa- 
tion - as a programmed instruction variable - has already 
been discussed (section 2.2 above). Evidently, theories con- 
cerning the effect of inlfermittent response information in 
PI cannot be based on the resuHs of studies of partial re- 
inforcement in animal learning, or human learning for thaf 
matter. 

I f effects similar to those obtained through partial re- 
inforcement, especially increased resistance to extinction, 
could be obtained in PI, "these effects would be at least 
partly negative. This point has been stated very clearly by 

Amsel : . . ^ ^ i. • 

"In training of problem-solving kinds of behavior, 
partial reinforcement would result in Pe^sistance 
of behavior which does pot. solve the problem (58:515). 

Although effects similar to those of partial reinforcement 
may be neither probable nor desirable in PI, intermittent re- 
sponse information may still be worth studying. It is possib- 
le that continuous key answer presentation, i.e. 100 per cent 
KF, as well as total elimination of key answers, i.e. 0 per 
cent KF, may have a negative effect on learning. Continuous 
response information may make the student, who knows that a 
key answer will always be presented, less eager to find out 
the right answer himself. And total lack of key answers, on 
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the other^hand, may have tiie obvious negative effect of al- 
lowing the student to repeat false responses, believing that 
they are correct (cf. partly similar argument by Moore 8s 
Smith, 41:200). 

Intermittent response information has been studied in six 
of the studies discussed here (3, 15/2 exp:s/, 30, 32 & 39), 
The comparison outcomes are sunsnarized tn table 3 . 
1 2 , (vhich also reports program length for each study. 



Table 3.12 Outcomes of comparisons between continuous 
and intermittent response information as 
to learning effect. 



Refe- Program Outcome of comparison betv/een KF 100% and 
rence length 75% 67% 50% 33% 25% 20% 10% 0% 



3 


140 










15a 


24 










15b 


235 


+ 








30 


177 










32 


177 




+ 






39 


1,144 











Note: Positive sign means that the effect of 100% l(F 
was better than that of the alternative ,KF 



Table 3.12 shows that 100 per cent KF has led to lower lear^- 
ning effect than 75, 67 or 50 per cent KF in six cases out 
of seven. Compared to the effect of 33 - 10 per cent KF the 
100 per cent KF has, hov^ever, led to better learning effect 
in four cases out of six. Apparently, intemrtttent response 
information has a positive effect, as campared to continuous 
response information, when less than half of the key answers 
are eliminated. When most of the key answers are eliminated, 
this seems to have a negative effect on learning. 

The interpretation of the data discussed here is, however^, 
complicated by tte:>var1attons in program length. As has al- 



f^^dy been demonstrated (section 3.5.1 above), key answers 
seem to havg no or even negative effect when programs are 
long, and positive effect when they are shorter. Two of the 
studies reviewed here may be interpreted in terms of program 
length. Krumboltz & Kiesler (30) found that 100 per cent KF 
led to better learning effect than did 20 or 10 per cent KF, 
and they used a 177 frame program, Lublin (39), on the other 
hand, used a 1144 frame program and found that 100 per cent 
KF led to signi.ficantly lower effect than 50 per cent KF. 

In three of the six studies reviewed here PI time data are 
reported. If PI time for each condition is divided by PI 
time under 100 per cent KF, the following quotients are ob- 
tained: 



Refe- 




PI time quotient for KF 


10% 


reYice 


100% 


75% 50% 25% 20% 


3 


1 


1.08 1.16 - 




15a 


1 


0.83 1.07 0.95 — 




30 


1 


« 1 .07 


0.98 



Thus, in four cases out of seven intermittent key answer 
presentation has increased the PI time by 7 to 16 per cent, 
as compared to PI time under 100 per cent KF. A similar ef- 
fect of total elimination of key answers (0 % KF) was, as we 
have seen, reported by Lublin (39). The elimination - to- 
tally or partly - of key answers in linear programs can, 
consequently, in some cases be expected to induce the stu- 
dents to spend more time on the program. If so, positive ef- 
fects on learning are a natural result of increased learning 
time. However, there is. no clearcut relationship between the 
PIT quotients presented above and the corresponding outcomes 
in table 3.12. On the one hand, an increase in PI time of 7- 
8 per cent accompanying 50 per cent KF, coincides with a 
slight increase in learning effect. On the other hand, two 
instances of longer PI time when response information is in- 
termittent (3 & 30; KF:s 25 and 20 %) are associated with 
lower learning effect than 100 per cent KF. 

This analysis of the effects of intermittent key answer pre- 
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sentation \r\ PI remains rather inconclusive, partly becaus-e 
experimental studies of this problem are scarce. It is note- 
worthy, though, that intermittent key answer presentation 
can have a positive effect, even if it has yet to be estab- 
lished under vvhat circumstances such an effect is most like- 
ly to occur. 



3.6 SUmiNG UP OF THE RESEARCH REVIEW 

The method employed in this review, cross tabulations on the 
basis of multidimensional classification of experiments, is, 
obviously, a valuable complement to the more traditional 
types of research review. 

The mo$t important problems and difficulties connected with 
the application of this method has already been discussed 
(section 3.2 above). That the method contributes to the de- 
monstration of co-ordination and standardization problems 
in educational research has also been pointed out. From the 
review reported here one can, for example, conclude that 
data on PI time should always be collected and reported in 
studies of the effects of programmed instruction. 

As regards the significance of response requirement and key 
answer frequency in PI, the review has primarily demonstra- 
ted that the influence of these factors as a rule can be ex- 
pected to depend upon different program and student variab- 
les. Theories about the superiority of one particular re- 
sponse requirement can, therefore, hardly be generalised to 
all program types or all categories of students. 

• y ■ ■ 

Some of the main reasons for adopting a non-dogmatic view 
of traditional PI features was admirably summarized by Reid 
already in 1964. He pointed out that Skinner- type program- 
ming was nothing new but simply improved control of learning, 
and continued: 

"when an interesting, meaningful subject is put into pro- 
grammed form the value may lie mainly in the improvement 
in exposition... Formal control by making 
the learner v/rite and check responses may even 
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occupy more time than it is 
worth. If he is intelligent and interested he can 
scarcely help formulating responses, and since the pro- 
gramme Viill make sure that his responses are very like- 
ly to be correct, he will have had his reinforcement be- 
fore he starts to write, far less check. ...with dull 
and disconnected material, e.g. spelling ... .the pro- 
gramming Itself may be less important and the control 
exerted by formal responding and checking ma^f be essen- 
tial for efficient learning." (115:159f; my emphasis) 

This review also leads to the conclusion that the evalua- 
/tion of different^PI features must depend upon whether the 
learning e Y f e c t or the learning e f f i c i e n c y. 
i.e. effect in, relation to time spent on the program, is the 
primary criterion of success. From a practical instructional 
viewpoint it seems quite clear that. the efficiency aspect 
cannot, without considerable loss of information, be neglec- 
ted 1n the experimental study of PI features such as respon- 
se requirement and key answer frequency. After all, the re- 
sults of instruction must be evaluated with reference to 
the price one has had to pay in order to reach them - and 
student working time is one of the most important cost fac- 
tors in any instructional situation. 



The Objectives of the Present Study 



The objective of the present study has been to help 1n pp^- 
vidtng a basis for decisions, in practical instructional 
situations* concerning the following issues: 

- What influence, if any, has the nature of the response 
requirement on learning effect and efficiency in pro- 
graiTBTied instruction using linear. Skinner-type programs? 

~ What influence, if any, has the presentation of key 
ansv/ers in the same respects? 

Are the effects of response requirement and key answer"^ 
frequency* if any, different at different age or grade 
levels? 

It should be noted here, that I have considered it more im- 
portant, in this context, to contribute to the basis for 
decisions regarding instructional practice, rath6i^ than to 
contribute to the formation of learning theory. This should, 
of course, not be taken to imply that these tv/o aims are ir- 
reconcilable; it is merely a question of priorities. 



64 



5 



Delimitation of the Object of Study 



5.1 THE INSTRUCTIONAL PROGRAM 

It was considered important that the program used in the 
study should treat a topic included in the official curri- 
culum of the Swedish comprehensive school. When the study 
was being planned such a program was constructed at the 
then National SchooVfor Educational Research in.Linkoping. 
The proaram is entvtfedJiIhe main parts' of the simple sen- 
tence". 'it treats /^su%cTr n!ameB^tnc1uded in the mother 
tongue curriculvmi^^or^JirtaaTe stage (grades 4 - 6) of 
the comprehens.'fve school. It is recommended that this topic 
be taught in grade 6, i.e. at the age of 12 years. Up to 
the beginning of grade 6 it would, therefore, be possible 
to find classes where this topic had not yet been the object 
of any systematic instruction. l\\\s would make 1t possible 
to use one and the same program at different grade levels. 

Tne main objective of the program is to teach the students 
how to distinguish and name the' two main parts of speech 
(subject part and predicate part) in simple sentences con- 
sisting of one head clause only, and to distinguish and 
name the main word (subject or predicate) within each part 
of the sentence. 

The program is linear and the original version calls for one 
student constructed response per frame: either one «r more 
missing words are to be filled in the program text, or a 
direct question is to be answered. In its final version the 
program has 241 frames. The version used in the P lot expe- 
riment of the present study was the second one. The program 
was revised once more on the basis of pilot study data, and 
the resulting, third version was used in the mam experl-_ 
ment of this study. 



It was attempted to construct a program that was easy to 
read and understand for students in grades 4 - 6. The read- 
ability of the final version has been measured using the 
most common Swedish readability index, LIX, which has been 
devised by Bjornsson (68) and has the following, simple 
formula: 

LIX ^ Mean number of words per sentence + percentage of 
long words; 

long words being words with more than sl?^ letters. The LIX 
value for the program Is 29, which, according to Bjornsson ^ 
(5B), Is the normal index for texts of medium difficulty for 
grade 4 students. Thus, students with a reading ability at 
grade 4 level or higher should be able to j^ead the program 
without great difficulty. ^ ^ y 

Data from the readability analysis can also be used to give 
some luca about the length of program frames. Since this* ' 
program Is entirely verbal, i.e. contains no pictures or 
diagrams, the length of its frames can be measured ln terms ' 
of v/ords or sentences per frame. The final version of the 
program has on the average 20 words per frame, and between 
three and four sentences per freme. Sentences are, thus, ' 
very short, with an average of six to seven words per sen- 
tence. ^ K » 

The subject matter of the program is of a kind usually sup- 
posed to be well suited for PI, since it is clearly delimi- 
ted and has a well defined structure, anc olnce'the ^ -rect- 
ness or falsfeness of propositions concerning the subje t ^ 
matter Cai» be objectively determined. Moreover, repetition 
using parallel but slightly different examples can easily be 
bunt Into the program. It should be pointed! out, however, 
that this program contains relatively few terms and con- 
cepts that are new to the students, and that it aims at the 
teaching of Identification of general structures rather than 
the memorizing of response terms. Thus, it belongs to the 
category of programs where the need for, or the advantage 
of overt responding can be expected to be rather small (cf. 
research review above, particularly section 3.3.2.3). 



5.2 THE STUDENT POPULATION 



As we have seen, the program chosen for the Pi^fent study 
should he suitable for use in grades 4, 5 and 5 of the com- 
prehensive school. A study of different PI features using 
this program can, consequently, include students from all 
three grades 4 through 6. Previous research in this area 
has mainly involved young adults or adolescents, vtz. Ame- 
rican college and high school students (cf. chapter 3 abo- 
ve). The need for additional experiments must, therefore, 
be considered greatest viith respect to younger students. 

In view of the objectives of this study the 10:12 year-olds 
in grades 4 - 6 are a suitable student population. Rippey 
(116) concluded from his review of research on overt and co- 
vert responding that the positive effect of overt responding 
tapers off between grades 4 and 6. Of course. Swedish stu- 
dents in these grades are somewhat older than their toien- 
can counterparts but grades 4 - 6 still provide a reasonab- 
le age range for a study of this kind. 

In terms of student population the present study has. then, 
been restricted to the middle stage (grades 4 - 6) of the 
comprehensive school .>e. to students at the age of IQ to 
12 years. The populatYon has. however, been further delimi- 
ted. Firstly, only normal class students are included, and 
not students in classes for mentally retarded, reading clas- 
ses or other special classes. Secondly, the study concerns 
only students vrfio do n o t 'master' the subject matter of 
the program prior to the experimental instruction; assuming 
that the concept 'to master' is operationally defined on the 
basis of a test of grammar knowledge. 

In the pilot study only grade 4 and grade 5 students were 
included, since this study was performed toward the end of 
a school year, when the subject matter in question had al- 
ready been treated rather thoroughly in most grade 5 clas- 
ses. 
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5.3 



PROGRAfI PRESENTATION AND LAYOUT 



Previous research by Malmquist a Srundin (104} had demonst- 
rated 

- that the presentation of linear Skinner-type programs by 
means of a simple, manually operated 'teaching machine' 
can be expected to result in the same amount of learning 
as presentation in the form of a programmed book, the so 
called horisontal section book^ and. 

- that the us^ of this machine usually will lead to longer'^^ 
.working time and greater risks of disturbances due to 
technical faults than the use of programmed books. 

Other experiments comparing the effects of machine and book 
presentations of linear programs have, according to a review 
by Goldstein & Gotkin (85), yielded similar results. On the 
basis of a review of research Eigen (78) has recommended 
that programmed books be used in initial field trials of in- 
structional programs. 

In the present ^tudy it was, therefore, decided that the 
program should b^ presented in the form of a horisontal sec- 
tion book, which will hereafter* for the sake of convenien- 
ce, be called 'book' or 'booklet'. In its final form the 
program consists of sin booklets, each containing some 40 
frames . 

Book presentation implies that the much discussed risk of 
cheating' is not eliminated. (It should be noted that many 
manually operated 'teaching machines' are not 'cheat-proof 
either.) Unless constantly supervised the student cannot, 
then, be stopped from checking the key answer on the follow- 
ing page before recording his own answer. Since constant su- 
pervising of students during PI is neither possible nor de- 
sirable in everyday school work, it was decided not to make 
any attempts at eliminating the risk of cheating through su- 
pervising in this study. Exact data regarding the amount of 
cheating that can be expected in situations similar to this 
one are scarce. Mills (110:130) has reported a 'cheating fre- 
quency of less than 5 per cent in programs of moderate dif- 
ficulty, but this was in a situation where the students only 
could cheat by copying from other students' answers. 



As some students may he tempted to 'outsmart the program . 
if they think they have discovered an unforeseen means ot 
doing so, the students were explicitly informed about the 
possibilities of 'cheating'. But at the same time they were 
informed that there would be a grammar test after the pro- 
gram, and that they could not expect to learn anything from 
the program unless they tried to solve the problems them- 
selves (cf. appendix 3: Instruction to the program). 

Aoart from a possible tendency to 'outsmart' the teacher or 
the program, cheating can usually be expected to occur only 
occasionally. Cheating should be no problem, if only the 
learning task 1s reasonable and well adapted to the abili- 
ties of the student. Moreover, if a student really does not 
know the answer to a question, it seans wise to look it up 
rather than to make wild guesses. In other words, what ac- 
cording to traditional teacher and parent morality is called 
cheating, is in many cases a sensible and rational problem- 
solving method. 

The mode of program presentation chosen for the present stu- 
dy implies, then, that the possible effect of key answers, 
presented to the student, might be the sum or the supposed- 
ly positive reinforcing effect of response ini'o^ation, on 
the one hand, and its 'cheat-stimulating' effect, v^ich tra- 
ditionally is supposed to bs negative, on the other. 



5 4 ALTERNATIVE RESPONSE REQUIREMENTS AND MODES OF 

RESPONSE INFORMATION 



Studies of the effect of different response requirements and 
modes of response information in PI- can include several al- 
ternative experimental conditions. The selection of alterna- 
tives for the present study has been made on the basis or 
the following conditions: 

(1) The effect of normal Skinnerian response requirement and 
mode of response information shall be evaluated. 

(2) Priority shall be given to alternatives to the Skinneri- 
an model, which are likely to increase learning effi- 
ciency. 
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(3) Experiences from previous research in the areas shalV 
be utilized. 

(4) ' It shall be possible to test all combinations of respon- 

se requirement and mode of response information in one 
program, without changing the form or content of the 
program frames. 

(5) The number of alternatives shall be limited to what is 
economically and practically feasible. 



Response requirements 

The response requirement of traditional Skinnerian PI. i e 
d^and for overt, student-constructed responses, must be in- 
f:rt:J^Z ?21 :107.) ^^^^^^'P^^^" ^^^^ <^ond1tion see, 

Multiplejchoice responses are of theoretical interest, but 
a comparison between constructed and multiple-choice respon- 
ses allows no evaluation of the effect of the demand for 
overt responding. 

A, from the point of view of efficiency, more interesting 
alternative to the OCR requirement (overt student-construc- 
ted response) is the absence of response requirement. This 

response requirement' has in severaV investigations been 
shov/n to be at least as efficient as the OCR requirement. 

•!u"5.^!^P°"^^ requirement cannot, however, be combined 
with different amounts of response information, and does 
sired here"°^ Pennit the kind of multifactorial design de- 

As alternative ^,o the Skinnerian OCR requirement was cho- 
sen, instead, the requirement which instructs the student 
to think out a response but not write it down, i.e. the de- 
mand for a covert, student-constructed response - the CCR 
requirement. This requirement demands no change in the pro- 

A )I a change m the Instructions given to the stu- 
dents. And it can be combined with different modes of re- 
sponse information. Since the CCR requirement as a rule 
leads to considerably shorter PI time than does the OCR re- 
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quirement (see section 3. 3,2. 4 above)* it is very interes- 
ting from the point of view of learning efficiency m PI. 

The following response requirements have, then, been cho- 
sen for the present experiment. 

Rl: an OCR requirement, i.e. the student is explicitly in- 
structed to write down^ in the response space, a 
response which he himself has constructed. 

R2: a CCR requirement, i.e. the student is explicitly in- 
structed to think out a response which fits into the 
response space, but not v^rite it dov/n. 

Note that R 1 and R 2. label the particular variants of 
the response requirements investigated in this study, vari- 
ants which are, in the end, defined through the wording of. 
the program and of the instructions to the students going 
through the program. Thus, OCR and CCR label types of re- 
sponse requirement, whereas Rl and R2 label individual in- 
stances of the respective types. 



5.4.2 Response information 



Here too, the response information mode of Skinnerian PI 
must be included. This means that the student is provided, 
after each frame, with the correct response to that frame, 
i.e. the key answer frequency is 100 per cent. (For Skin- 
ner's own description of this mode see for example 121:99.) 
This mode of response information will hereafter be label- 
led F 1 . 

Starting from theFl alternative defined above, the response 
information can vary as to type^, delay in relation to the 
responding^ and frequency. The most corranon alternative to 
Fl in terms of type is to tell the student whether his re- 
sponse is right or wrong. When responses are student-con- 
structed, this type of information is, however, impossible 
unless one has access to very sophisticated machinery for 
the evaluation of student responses prior to the presenta- 
tion of response information. Obviously, this is not possib- 
le v^hen the program is presented in book form. 



Delay in the presentation of response information is also 
incompatible with book presentation, since a student going 
through a programmed book is always entirely free to deter- 
mine himself the delay between his responding and his con- 
sulting the key answer - if he at all consults it. Besides, 
there is nothing that indicates the possibility of a gain 
by postponing the presentation of information that will al- 
ways be presented in the end. 

Key answer frequency, however, can easily be varied irres- 
pective of the program presentation mode and whether respon- 
ding Is overt or covert. Previous research indicates also 
that yariations in the key answer frequency in PI may iji- 
fluence learning (section 3,5 above). Fl with its 100 per 
cent key answer frequency Is one extreme on the continuum 
of key. answer frequencies. Possible alternatives are all 
frequencies below 100 per cent. 

In order to judge whether the presentation of key answers 
at all has an influence on the learning effect of PI, or on 
the PI time, the effect of totally eliminating the key ans- 
wers should be studied, i.e. the key answer frequency 0 per 
cent. Previous experiments have shown that this alternative 
- especially when programs are fairly long - may result In 
at least as good learning as the 100 per cent frequency 
(Fl), This 0 per cent key answer frequency, hereafter label- 
led FO, also allows greater liberty in the layout of pro- 
grammed books. The so called vertical book, where frames are 
read one after the other down to the bottom of the page, 
can, for instance, be used without a masking device, Milan 
& Bernath (109) he^ve pointed this out, and claim that they 
have been able to construct a better program thanks to the 
elimination of key ansv/ers. 

If the correlation between key answer frequency and learning 
effect in PI is linear, the two alternatives Fl and FO would 
be sufficient for an evaluation of the direction and degree 
of this correlation. As was demonstrated in the research re- 
view (section 3,5.2 above), the possibility of a non-linear 
relationship between key answer frequency and learning ef- 
fect should, however, not be excluded. To test the lineari- 
ty of the correlation the effect must be measured at not 
less than three points along the key answer frequency con- 
tinuum. 
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and 



\lnce there is no really safe ground for formulating a hy- 
ToSesis concerning the^onn of a possible "on- inear corre- 
latinn between tey anSwer frequency and learning effect (cf. 
seSfon 1 ablJe). the thi?d alternative has si jp y been 
located halfways between the extremes, i -.^t 50 per cent 
kev answer frequency. If the 50 per cent alternative, here 
Slltd F 5 is to yield a. basis for estimating the linearity 
"f"tS cor'r^latio'n in question, it rl^P^^^,^P|o ^.^^ ^t 
to reduce the key answer frequency from t° Pe. cent 
by means of either a random elimination of half the key ans 
wers or elimination of every other key fswer Both th s 
procedures have been used in previous studies (32 & 39;. a 
there is no reason to expect them to differ in their influ 
ence upon the learning effect. 

From the practical viewpoint of real -life instruction both 
these procedures of eliminating key answers seem, however, 
unlui table, since they do not take account of the structure 
of the individual program, i-.e. neither Procedure can be 
exoected to yield response infomation that is optimally 
sp ed throughout the program. A variant J^sed upon some 
kind of systematic variation, adapted to the ndividual pro 
gS. between frames with and frames without key answers 
following them would therefore be more appropriate in a stu- 
dy with a practical instruetional objective. 

In the present study the alternative with 50 Per cent key 
ahswers has the following form. The program was div ded n- 
to sequences of 10 to 20 frames, which are relatively 'clo- 
sed Si?"' as to content. Within each sequence key answers 
a?e.giJen after the first 50 per cent of the ;^^ames. but not 
after the remaining frames. This variant is adapted to the 
Droaram used in the study, and at the same time it can be 
fSed afa scale point in a variable, viz. the l^roportion 
S? f?aSes. counted from the first, in each sequence that are 
followed by key answers. 

The different experimental conditions as regards Infomation 
about corfea responses included in the study has. then, 
been 

Fl: key answers presented after 100 per cent of the fra^K^s 
in each program sequence; 



Er5: key answers presented after 50 per cent of the frames 
1n each progrtim sequence; and 

FO: key answers presented after 0 per cent of the frames 
1n each program sequence. 

Note that previously employed designations such as 100 per 
cent KF, 50 per cent KF, denote different types of response 
Information modes. The designations introduced here, Flp 
F.5, and FO, denote only the individual instances of these 
types that are studied in the present experiment. Like Rl 
and R2 (section 5.4.1 above), these alternatives are in the 
end operationally defined by the material, the instructions 
etc. used in this study. It is especially important to dis- 
tinguish betv/een the more general concept '50 per cent key 
answer frequency' and F.5, since F.5 is only one out of se- 
yerat possible modes of response information that provides 
key answers after, on the average, 50 per cent of the frames. 



The Experimental Design 



The experiment has the objective of analysing the effects 
of the variables response requirement (R) and key answer 
frequency (F) in at different grade levels (G). Further- 
more, it may be of interest to study difference between se- 
xes (S) as to these effects. The experiiitent was , therefore, 
planned so as to allow a complete multifactorial design with 

L(R) X L(F) X L(G) x L{S) cells, . . 

Where L(i) is the numbers of values of the variable i. 

It has been shown (chapter 3 above) that the effect of dif- 
ferent response requirements or key answer frequencies may 
be dependent upon other factors in the learning situation. 
An interaction between response requirement and the stu- 
. dents' grade level is, for example, possible. The mere pos- 
siMlity of such interacttons make multifactorial studies 
of the variables involved desirable. The choice of this par- 
ticular multifactorial design does not imply, however, that 
specific hypotheses have been formulated, prior to data col- 
lecting, concerning all Interactions of the first, second, 
and third order that can be tested in such a design. Jhe use 
of a complete multifactorial design has partly the aim ot 
creating a basis for the fomulation of new hypotheses. 

Most previous experiments concerning response requirement 
and correct response infortnation in PI have not employed 
multifactorial designs. The use of very simple research 
designs, can lead to oversimplified conclusions concerning 
the var^ibles involved. This point Vas made already in 196,1 
by Lumsdaine, who emphasized that i «^ 

. "the need to seek contingent rather than absolute gene- 
ralizations implies the frequent use of factorially 
designed experiments in which the operation of one 
variable can be observed as it interacts with other 
variables." (98:500) 



The question of which interactions and v/hich sets of ortho- 
gonal comparisons vvill be of particular interest is further 
discussed in the chapter "Hypotheses" below. The number of 
cells in the complete experimental design is 

2x3 X 3 X 2 = 36, 
since 

L(R) = 2 ; i.e. Rl, R2 
L(F = 3 \ i.e. Fl, F.5, FO 
LfG) 3 • i ,e. grade 4, 5, 6 
L(S) = 2 ; i.e. male, female. 

If, for example, 8 replicates per cell is considered de- 
sirable, the minimum number of subjects that must be in- 
cluded in the experiment is 288. 
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Hypotheses 



The principle of testing only such hypotheses as are fomu- 
lated prior to the data collection will not be strictly 
applied in this study. My primary aim in the data analysis 
has been to interpret the data at hand as objectively and 
exhaustively as possible. A strict application of the prin- 
ciple mentioned above means that any new idea, arising du- 
ring the course of an experiment, demands a wholly new ex- 
periment for its testing. The alternative, i.e. no strict 
application of this prtnciple, means instead that the possi 
bilities of data analysis are restricted primarily by the 



"According to the likelihood principle, data analysis 
stands on its om feet. The intentions of the experi- 
menter are irrelevant to the interpretation of data 
once collected, though of course they are crucial to 
the design of experiments." (77:239) 



belol are discussed the hypotheses which, in view of avai 
able\nowledge, have been considered reasonable to formul 

7 J HYPOTHESES COrCERNING RESPOr^SE REQUIREMENT 
7.1.1 The influence of je ^me req uirenient on lA^rnM 



The program used in this study can be considered long enough 




eWecT 



77 



(mare than. 200 frames) for a positive effect of the overt 
response requirement (Rl) to show (cf. section 3.3.2.1 
above) . , 

The subject matter programmed (elementary analysis of sen- 
tence structure) is of such a nature that the positive ef- 
fect of the overt response requirement (Rl) can be expected 
to be small or even nqnexistant (cf. 3.3.2.3 above). The 
positive effect of Rl , as compared to R2, is likely to be 
greatest for the students who are youngest and have least 
initial knowledge, i.e, the grade 4 studetits, and the effect 
is likely to be smallest for those who are oldest and have 
most initial knowledge, i.e. the grade 6 students (cf. sec- 
tion 3.3.2.2 above). 

In the present experiment one can, then, expect 

that the learning effect and the retention is, on the 
whole, significantly higher under Rl (overt response require- 
ment) than under R2 (covert response requirement); and 

that the positive effect of Rl compared to R2 in these 
respects increases when the grade level of the subjects de-* 
creases from grade 6 to grade 4. 



7.1.2 The influence of response requirement on Pt time 



In view of the results of previous research (see especially 
section 3.3.2.4 above), one can expect in this study 

that PI time is shorter under R2 than under Rl» irrespec- 
tive of the variation in other design variables (key answer 
frequency, grade level, and sex); 

that the increase in PI time following Rl , as compared to 
R2, is of the order of magnitude 50 - 100 per cent; and 

that this increase In PI time is smaller the higher the 
grade level of the subjects within the interval grade 4 to 
grade 6. 
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From the hypotheses listed above it follows 
that R2 (covert response requirement) on the whole leads 
to better learning efficiency than Rl (overt response requi- 
rement); and 

that this superiority of R2, compared to Rl. is more 
marked the higher the grade level of the students within 
the Interval grade 4 to grade 6. 



7.2 HYPOTHESES CONCERNING KEY ANSWER FREQUENCY 

7,2,1 The influence of key , a nsyar frequenc y on Igarnjng. 
effect 

When programs are long (more than 200 frames), 100 per cent 
l2y answlr frequency, in combination with demand for overt 
resDonding, seems to have a predominantly negative effect 
Sa?ed to the effect of 0 per cent key answer frequency 
(S section 3:5.1 above). The effect of key answers when 
covert responses are required have rarely been studied, but 
these effects may well be different from the corresponding 
effects under overt response requirements. If one assumes, 
as do Leith (35) and Anderson (59). that overt responses 
have an attention-focusing function; this/unction may. in 
the absence of overt responding, be transrerred to the key 
answers. The elimination of overt ^i^sponding as well as key 
answers may. therefore, be expected to result n ^ negative 
effect, especially for the younger students, who probably 
need attention-focusing devices in PI most. 

Regarding the influence of the key answer frequency upon the 
learning effect or the retention it -an. then, be hypothe- 
sized 

that under Rl decreasing key answer frequency, from Fl 



to FQ, results In increasing learning effect or retention; 
and 

that under R2 decreasing key answer frequency p from FT 

to F0> results in decreasing learning effect or retention. 



7.2.2 The influence of key ansv/er frequency on PI time 

In view of v^hat has been established by previous research 
(section 3.5 above) it seems reasonable to hypothesize 

that PI time, at least for students v/orking^ under Rl » 
{overt response requirement), inc^'eases as key answer fre- 
quency decreases from Fl to FO. 



^•2.3 The influence of key ansv/er frequency on learning 
efficiency ^ ~^ 



On the basis of the hypotheses concerning learning effect 
and PI time listed above* one may expect 

that under Rl (overt response requiretnent) the efficien- 
cy is approximately equal for the different key answer fre- 
quencies, Fl, F.5, and FOj and = 

that under R2 (covert response requiretnent) the effici- 
ency decreases when the key answer frequency decreases from 
Fl to FO. 



7.3 HYPOTHESIS CONCERNING THE NORf^AL SKINNERIAN CON- 

DITION, RlFl 



The normal condition in programmed instruction according to 
the Skinner model is the combination RlFl, i.e. overt re- 
sponse requirement and 100 per cent key answers. 
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If this normal condition is also, as many assume » the opti- 
mal condition, one may fot^late the hypothesis 
that learning effect, and learning efficiency under the 
condition RlFl is greater than the effect and efficiency 
under any other combination of R and F alternatives. 

Contrary to the other hypotheses formulated here this is 
merely a control hypothesis, and not something the author 
believes to be true. 
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Variables: General Definitions 



The design variables R, F, G, and S have already been de- 
fined in chapters 5 and 6 above. Here are presented variab- 
les \vhich are included in the- study as variates or as co- 
variates. 



8.1 ^ VARIATES 



The experiment has the objective of studying how the design 
variables, especially R (response requirement) and F (key 
ansvver frequency), influence PI time, learning effect, re- 
tention and learning efficiency. 

These four variates (dependent variables) are defined, in 
general terms, as follows. 



PI time: 



gross time, from start to completion 
0^ last frame, for going through each 
program booklet, rounded off to whole 
minutes and summed for all six book- 
lets 



Learning effect: 



increment in performance, measured by 
means of a special test, from pre- 



testing before the PI to posttesting 
no. 1 the day after completion of the 
program 



Retention: 



ditto from pretesting to posttesting 
no. 2 about eight weeks after comple- 
tion of the program 
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Learning efficiency: learning effect in relation to PI 

time (a more precise measure has not 
been defined beforehand - see sec- 
tion 3.3.2,4 above). 



8.2 COVARIATES OR CONTROL VARIABLES 



The analysis of effect and efficiency, as defined above, 
presupposes the existence of a measure of the students' ini- 
tial knowledge of the subject matter treated in the program. 
This measure can then be used either in the computation of 
a performance increment (posttest - pretest) or as a cova- 
riate. . • 

A subject's value in a criterion variable will, to some 
extent, depend upon his intelligence. It has been contended 
that in PI learning depends to a very small extent - or not 
at all (I) - on the general intelligence of the student. 
This contention is, however, to tfie best of my knowledge 
not supported by unequivocal empirical evidence, A previous 
experiment by Malmquist & Grundin (107) showed a significant 
correlation between IQ of students and learning effect. In 
the present study this correlation can be expected to be 
rather high, since the learning task demands a substantial 
amount of logical thinking, especially in the application of 
a sentence analysis method, learned from certain examples, 
to new examples. The verbal and the logico-inductive or 
reasoning factors of intelligence should be particularly im- 
portant. 

The following covariates or control variables are, then, in- 
cluded in the ^tudy. 



Initial knowledge: 
Verbal intelligence: 



performance, as measured by a spe- 
cial test, the week before the PI 

the result on an intelligence sub- 
test (DBA:2), administered the 
week before the PI 
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Inductive tntelllgence: the result on another intelligence 
) subtest (DBA:3), administered the 

vjeek before the PI. 
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The Pilot Study 



9A OBJKriVE 



The main objective of the pilot study v/as to control the 
feasibility of trie experiniant as regards design, admlnistra 
tion, etc, and to test all materials, such as program, in- 
structions and tests. Fhe pilot study did not have the aim 
of providing data for the tasting of the hypotheses formula- 
ted for this study. If the statistical analysis of pilot 
study data has, nevertheless, been rather thorough » this is 
primarily because this analysis was expected to guide the 
choice of dota analysis methods to be used in the main stu- 
dy. 



subjects in tntt p^^ot study nere stuaents -In the then Matio- 
nal Scnoel for Zoucauonal Researcr^ tn LinKoping. Since it 
v/as c^^tU26 cut diiPing a sprifig ttmv,, the subject matter in 
•^i^-stion had tiiroacfy been treated in the grade 6 classes, so 
t.-Q pilot 'sta'CJy ^as li^^Hed to twj grade levels, 4 and 5. 

Wit^Mn ea^:r: of t:..K,r ^^i:e*''irm'iza't clas^fe^; thR students were 
aiv;^je1 t eAfc^r'^nottal ^^riups ;3y rneans of a random pro- 
CQ^ure {lot ^rd^inf}, TrH don^ sej^av^jtely for boys and 



ta1 groups. 



Group Exp« G^aae ^ ^' j''" ' 





cond. 


M 


r 




1 


Rin 


t: 


»^ 




2 


R1F.5 




h 




3 


RIFO 




i 




4 


R2F1 








5 


R2F.5 








5 


R2F0 









Total ?j ./^* -i^ 



9.3 T^^vr') A':^ 



this studytf t^ie u l-^t. . ^ 
This vernlf:^ ^-u^^'. : 
and 1 't^U^: 

The follOiV:-:-; '.^-r ^ u-^^'^' ■■ 
jects: 

result : ■'^■•'^ ~" 

total f: 

tit.n ct t^'^^ ■ - ■ * 

For sutjOi:' . -'^ ^ ■ « 

fiumfee^ ^■; ^ , . ■ : 
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9.4 LESSON PLAN 

The pilot experiment comprised 11 lessons as follows. 

Lesson 1, 2: Pretesting (DBA tests and PUSL test). 

Lesson 3: Training of the mode of working with the 
program (through administration of a pre- 
program). 

Lesson 4-9: Programmed instruction, one booklet each 
lesson and not more than one lesson each 
day. 

Lesson 10: Posttesting no. 1, the day after lesson 9. 

Lesson Hi: Posttesting ncv, 2» six weeks after lesson 
10. 



9.5 IMPLEMENTATION OF THE PILOT STUDY 



Lesson 1: The intelligence tests DBA:2 and DBA:3 were 
given in that order in strict accordance with the instruc- 
tions in the test manual. Experimental class teachers func- 
tioned as testers, but classes had been interchanged so that 
no teacher tested his own students. 

Lesson 2: The PUSL test, constructed especially for 
the purpose of this study (for detailed description see chap- 
ter 11 below), was administered according to its written in- 
structions by the same testers as in lesson 1. 

Lesson 3: Students were introduced to the mode of wor- 
king with the program to which they hade been assigned^ A 
special introductory lesson was held for each experimental 
condition (groups 1 through 6), so that each student got on- 
ly the-instructions relevant to the condition he was assig- 
ned to, and all students assigned to the same condition got 
the same instructions, irrespective of their grade level. 



All instructions v^rere given orally by one Instructor for 
groups 1 - 3 and another for groups 4-6. 

Lesson 4 - 9: Once the students had learnt the mode . 
of working for each particular group, it was not considered 
necessary to have all six groups work in separate rooms, but 
only to separate those writing answers in their booklets 
from those not v/rit1ng them. Within each gr.ade all Rl groups 
were brought together in one room and all R2 groups in an- 
other. Instructors were the teachers of the four experimen- 
tal classes. Each teacher supervised a group of students of 
the same size as a normal class. 

The teachers were given detailed instructions in writing a- 
bout what to say and do during each lesson. The students we- 
re given short written instructions, reminding them about 
the mode of work prescribed for their group. Each lesson 
started with a short oral introduction by the teacher/inst- 
ructor. 

In order to keep those students occupied, who, had completed 
their booklets, additional tasks were given in the form of 
arithmetic drill exercises. The students were told to start 
with these exercises as soon as they had completed the pro- 
gram booklet. A lessori in this series (nos. 4 - 9 con- 
sidered finished when the teacher observed that all students 
had completed their program booklet. 

L'^e s s 0 n 1 0: The day after lesson 9 the PUSL test was 
given once again with a slightly different instruction, by 
the same testers as during lessons 1 2. The small diffe- 
rence in test instruction was motivated by the fact that the 
students now had gone through the program and were supposed 
to have learnt the subject matter in question. 

Lesson 1 1: Six weeks after lesson 10 the PUSL test^ 
was administered for the third time with ^h? the same inst- 
ruction as during lesson 10, except for a "'i"°^^^;^"9% " 
the tester's introduction due to the rather long time elap- 
sed since the previous testing occasi^on. 
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9.6 DATA ANALYSIS PROCEDURE 



^ 9.6.1 Final set of date. 



Of the 116 students in the four experimental classes 115 be- 
longed to the population defined as students in grade 4 or 5 
scoring less than 30 on' the PUSL test given as a pretest 
(cf. definition of population in 5.2 above). 

In 'all 84 students had been present during all lessons in 
the experiment. The number of students with no missing data 
was* not less than 6 and not more than 9 in any subgroup in 
the R X F X G design. In order to facilitate data analysis 
in the multifactorial design R x F x G subjects v^ere exclu- 
ded by means of a random procedure until exactly 6 subjects 
rel^ained in each subgroup. 



9.6.2 Analysis of variance method 



Preliminary analyses of pilot study data indicated some ra- 
ther marked interactions between sex of subjects and experi- 
mental conditions. Analyses of the R x F x 6 x S design with 
24 cells were therefore desirable. As the distribution of 
boys and girls varied from subgroup to subgroup, the number 
of subjects per subgroup was not constant in this design. E- 
qual number of subjects per cell could have been obtained 
only by reducing the number of subjects per cell to 2 and 
reducing the total number of subjects in the analyses from 
72 to 48. In order to avoid this, variance analyses of the 
design R x F x G x S were performed by means of a method al- 
lowing unequal numbers of subjects per cell (124:381). This 
method involved analysis of cell means instead of cell sums, 
,and also the computation of an approximate error mean square 
value on the basis of the mean square within cells, which is 
computed in the usual way. The formula for the approximate 
error mean square is 

4 = (|(l/n^.)/k)-(Vdf^,) ; 



er|o 
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vjhere = number of Ss in cell U 
k = number of cells, 

S = the sum of squares within cells, and 
df^]' = the degrees of freedom of S^. 

Principally tM same method of analysis as has been descri- 
bed above was employed for the purpose of covanance analy- 
ses The only difference is thatthe cell means and the sum 
of squares within cells of the variate are adjusted TOr re- 
gression on the covartates included in the analysis. Adjus- 
ted means and sums of squares have been computed following 
this procedure: 

(1) Cell means, square sums and product sums totally (T) and 
between cells (b) are computed as in conventional analy- 
sis of variance (see e.g. 124: chapter 15). 

(Z) Square and product sums within cells are computed on the 
basis of the relationship S(w) = S(T) - S(b). From the 
resulting matrix square and product sums within cells, 
correlations and r'egression coefficients within cells 
are computed in the usual way. The multiple correlation 
coefficient is then computed using the formula 

^y.i2..k = g(^yj'^yj-i2..k) • 

where b^, no . = the partial regression coefficient for 
yj.i^:..K covariate j, expressed in la scale 
free* measure (124:326), and 

V = the number of cov^riates. 

(3) Adjusted cell means for the variate (Y) are computed 
from the fonnula 

where j = a covariate, 

V = the number of covariates, 

i = a cell , and . ; 

b = the partial regression coefficient within 

cells. 

(4) The sum of squares within cells of the variate is final- 
ly adjusted according to ttie formula 
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9.? ANALYSIS RESULTS 



9.7.1 Different modes of analysing posttest results 



Four different ways of analysing posttest data by means of 
variance analysis have been studied. Variates and covariates 
Included in each type of analysis are shown below. 



Alt. 


Variate 


Covariate(s) 


1 


POST 




2 


POST - PRE 




3 


- POST 


D2 


4 


POST 


D2, PRE 



Alternative no. 1 analyses variation between 
gells in terms of performance level after the PT, Conclusi- 
ons regarding learning effect and efficiency can only be 
drawn under the assumption that there were no differences 
between the groups as to performance before the PI. This as- 
siimption is, of course, reasonable in view of the random 
distribution of subjects on experimental groups. 

Alternative no, 2 analyses variation between 
cells in terms of the increase in performance from prestes- 
ting to posttesting. Differences between groups in pretest 
performance are accounted for. The analysis presupposes, 
hov/ever, that the coefficient of regression v/ithin cells e- 
quals ], If tt does not, the estimate of the error mean squa- 
re is not minimized (cf. 76:295ff). Also, the reliability of 
a difference betv/een two scores can be very low, if the sco- 
res are highly intercorrelated, as has been pointed out by 
Guilford (87:393f) in his discussion of the Mosier formula 
for the reliability of a difference score (111). Unless pre- 
test scores are close to zero, pretest and posttest scores 



12 



can be expected to he rather highly correlated. 

Alternative no. 3 analyses variation betv/een 
cells in terms of performance adjusted for regression on one 
covariate. D2 Is chosen simply because correlation data sug- 
gest that it is the best predictor of posttest scores (POSTl 
as v/ell as P0ST2). This analysis, then, accounts maximally 
for the criterion score regression on one single covariate. 

Alternative no. 4 differs from no. 3 in one 
respect only: the number of covariates is increased to two. 
The covariates D2 and PRE (pretest score) are chosen befcause 
they constitute the best set of two predictors of posttest 
results. 

Coefficients of correlation within cells in the design R x 
F X G X S between the various covariates and variates are 
rendered in t a b 1 e 9.2, Coefficients of regression 
within cells for one and two covariates respectively are 
rendered in t a b 1 e 9 . 3 . 



Table 9.h Coefficients of correlation within cells in the 
design R x F x G x S (df = 45). 



D2 D3 PIT POSTl P0ST2 

PrT"^ .260 .254 -.053 .541 .468 

D2 — .395 -.174 .698 .536 

03 - -.208 .45^7 .340 



PIT 
POSTl 



-.320 -.383 
789 



A simple measure of the relative efficienc/ of the different 
analysis methods discussed here can be obtained by means ot 
comparing the error variances. Alternatives 2, 3 and 4 re- 
sult in successively decreasing error variance compared to 
alternative no. 1. If error variances are expressed as per- 
centages of the alternative no. 1 error variance, the folio- 



wing values are obtained. 

Variable Alt. 2 Alt. 3 Alt. 4 



POSTl 
P0ST2 



70 
81, 



53 
73 



39 
62 



Table 9.3 Coefficients of regression within cells and R - 
values for POSTl and P0ST2, when one and two co- 
variate§'^respectively are included. 



Covarl- 
ates 


Variate 


^D2 


^PRE 




1 


POSTl 


.977 




AB7 


1 


P0ST2 


.676 




.287 


2 


POSTl 


.837 


,679 


.626 


2 


P0ST2 


.561 


.558 


.402 



Judged by the magnitude of the error variance^ alternative 
no. 4 seems clearly superior to all the other alternatives. 
X^^^^ It also appears reasonable that a study of the influence of 
""'"'certain program variables in PI should, as far as possible^ 
eliminate variation 1n posttest performance caused by diffe- 
rences betweeen subjects in intelligence and initial know- 
ledge; unless, of course, these variables are included as a~ 
nalysis of variance variables. 

Therte has lately been a great deal of discussion of the use 
and msusQ of covarlance analysis (ANCOVA). Berglund (66) 
and Aiken (57) caution against indiscriminate use of ANCOVA^ 
and Elashoff treats the problem quite thoroughly in a 1969 
paper (81). Aiken recommends blocking as an alternative to 
ANCOVA, but Cox (73), on the other hand, found that ANCOVA 
is clearly preferable to blocking when r - or R - is large, 
especially if it Is greater than 0*8, which is the case in 
the present study. And E1ashoff»s conclusion is that 
"violation of the assiimptlon /underlying ANCGVA/ of homo- 
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geneity -Sf regression or homogeneity. of variances 
be less serious if individuals have been assigned 
to treatments at random and the x" variable has a normal 
distribution" (81:396). . ^ « • 

The pilot study data v/ill not be used i;or the purpose of hy- 
pothesis testing. Vrom the point, of view of design feasibi-* 
lity it is, however, interesting to note that ANCQVA accor- 
ding to alternative no, 4 (s^ above) has yielded a number . 
of significant F-^quotients - in spile (ffv the small number of 
degrees of freedom within cells, i.e. 48» When the variate 
POST! -was analysed significant F-quotients were obtained for 
the main factors F and G, and for the ijiteraction R\x S. The 
P0ST2 analysis yielded significant F-quotients for R and G, 
and for the interaction R x F x S^i 

The analysis of learning effect and retention* data from the' 
, pilar study demonstrated, then, that the design used in thie 
study can be considered well suited for its purpose of cje- 
monstrating differences between response requirements and 
between key answ^ frequencies as to their effect on ^lear- 
ning In PI, i 



9. 7.2. 



TI tinte 



I. 

When students have a free choite of working, tempo, time va- 
riables such as reading time or working time often have a 
positively skewed distribution. This usually implies a cor- 
relation between the mean and the standard deviation for 
different subgroups and^ thus, heterogeneity of/ variance be- 
tween groups. As aVule this can be corrected \by means of a 
square-root or 
e. t4me in minu 



logarithm transformation of the raw 
tes or seconds (cf. 76:128-1^31). ^. 



score, i. 



As f i g u r e 9 . 1 shows/, the distributio,h of PI time 
scores in the pilot study was\ however, only moderately 
skewed. The variance analysis could, therefore, be performed 
directly bn the raw scores^,, PI time in minute^ (cf\ 76:132). 
This v^iance analysis, yielded a number of sij^nificant F- 
quotients: for the main factors R, F and G, e^nd for 1;he in- 
teractions R X G, 6 X S and R x 



F X G. Jhe pilot study 



has. 
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Figure 9.1 Distnbutiorf of PI time (PIT) scores for pilot 
study subjects (N 72). 



then, given ample additional evidence for the need to take 
PI time into account in studies of this kind (cf. section 
3ij6 above)'. 



- "9.7.3 Learning efficiency 

« c 

As has already been pointed out (section 2.5 above), lear- 
,,ning efficiency measures such as performance divided &y^vl 

time are -usually 4iot acceptable. Under certain circumstances 
"it seems » however, reasonable' to try Hnd dete*miin6 learning 
efficiency by means of a covariance analysis, where learning 
effect measures are adjusted for regression on PI time. 

A suitable learning effect measure (^) in this study is, ac- 
cording'to analyses reported above (section 9.7,1)^ 



E =^P0ST1 - b^^{D2 - - bpj^^(PRE - Wl) 

where .b isj-the coefficient of^partial regression within 
cells in the analysis of variance design. 

If learning efficiency (EFF) is defined as effect adjusted 
for regression on PI time, then 

EFF =^E - bpj^(PIT - TTT). ^ 

When efficiency is expressed as a function of effect and ti- 
me - EFF = f(E,PIT) - the definition of efficiency (section 
2.5' above) presupposes that, for a constant E, EFF increases 
when PIT decreases and vice versa,, If 

• ^ f(E,PIT) = E - bpj^(PIT - TTT) , ^ 

th^n the b-value for PIT mus^t' be positive, i.ec*the correla- 
tion between learning effect (E) andPI time (PIT) must^bfe 
positive, which in its turn tmplies that the partial -corre- , 
lation between p6sttes|: score (P0ST1) and PI time with D2 
and. PRE scores held constant, is also positive. , 

f ■ 
For the pilot study subjects in grad^ 5 PI time was about 
the same for both response requirements (overt anfl covert). 
This was totally unexpected, and there was reason to believe 
♦that it was an incidental instructor effect/which could be 
eliminated in the main study. Computation and analysis of 
efficiency (EFF) data as defined above was, therefore, re- 
stricted to grade 4 subjects. . u 

Partial regression coefficients within cells (b) were'compu- 
ted from the matrix of zero-order correlation coefficients. 
For the covariate PIT was obta-ined 

^PIT = +0.0704., 
which means, that the requirement stated Sbove is satisfied. 

The-^variance analysis of the learning efficiency variable, 
EFF, yielded no significant F-quoti^nts. It is interesting 
to note, though, that whereas the observed effect (E) was 
greater for Rl than for R2 groups in grade 4, the efficiency 
(EFF) w^s greater for R2 than for Rl groups. This demonstra- 
tes clearly that the outcome of ^comparisons in terms of 
learning efficiency may be quit6 different from the outcomes 
of learning ^ef feet comparisons. 



The relative learning efficiency. of the different response ' 
-requlVements can ^ilso be estimated by means of .the;i^onclu- 
sfon chart suggested in the literatu^se review (section 3.3. 
. 2.4 -dtpowe). Due to the significant interaction R x G in the" 
variable ^PIT (sect-ion 9.7.2 above) grades 4 and 5 should be^ 
treated -separately. As tn the previous analysis, effect*(E) 
Is pdmest score (POSH) adjusted for regression on verbar ■ 
intelligence (D2) and pretest score (PRE). The application 
of the corfclusion chart yields* the following results.^ 

I G r a d e 4 Y E(-R1 ) = E(R2) -and ' ' 
PIT(RT) > PIT(R2), thus 
• EFF(Rl)' < EFF(R2). ' , ' 



Grade 5 : E(R1) = E(R2) a n d - 

PIT(Rl) = PITCR2) thus 

EFF(Rl) = ^FF R2 . • . ^ . ^ 



Note that here means 'not significantly different from'. 
The conclusion would be, if conclusions were to be drawn 
from pilot study restilts, that overt response requirement 
(Rl) leads to lower effici'ency than co'yert^response require- 
ment in grade 4, whereas, in grade-5 both- requirements lead 
to the same efficiency. 



9.7.4 ^ Student response errors in the.program 

For all students in Rl grbup,s, i.e. those who were told to ' 
write down their answers in the program booklets, the sums 
of incorrect and omitted responses in the booklet^ were com- 
puted/This: ^^ariable (ERR) has a markedly skewed distribu- 
tion, as can be seen in figure 9 2 . Square-root 
transformation according to th^ formula 

ERT = (ERR +0.5)^/^ ^(cf. 76:128ff) *a 

gives a roughly symmetrical distribution, as figure 
9 , 3 shows.' The analysis of the variable ERT has comprised 
correlation and xovariance analysis. The following correla-' 
tion coefficients within cellos have been computed: 



;0 



\ 

, PRE ERT 



D2 .148 -.696 ' • . 

PRE. -;560 . ^ 

This gives a^'squared coefficient of multiple correlation be- 
tv^een ERT, on the one hand, and D2 and PRE,- on the other, of 
0."fe98, which means that the degree of covariation between . 
ERT and D2^and PRE is 70 per cent. 

A variance analysis' of ERT cell means adjusted for> the r^g- 
r.jssion.on. D2 and PRE yielded significant^F-values for the 
main factors F and G, and for the interacti£>n F x G. The 
^number of errors (ERT) increased with 'decreasing key answer 
frequency (from Fl to FO). Thts could be due to the fact 
that the students werfe allowed to correct tijeir responses 
during their work with the program, since key answers could 
be utilized for the purpose of correcting student answers. 
This indicates that data regardjfig the frequency of such 
corrections shg^ld be included in studies of this type.''^ ; 
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Figure 9,2 Distribution gf pilot study subjects in. 
thq variable ERR (N = 36)/ 
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Figure 9.2 Distribution of pilot study subjects in 
^ the variable ERT (N =^ 36).-- 
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Revision of Program, Design etc. Prior to the Main Study 





The pilot experiment findings indicated that the main study 
cou'ld'be carried out'with the same general design, after 
affinal revision of the instructional program, the instruc- 
ti.ons to the students and the teachers, and the criterion 
teit - the PUSL test. The pr^ogram was revised on the basis 
of error response rates in the pilot experiment and sugjges- 
1>ions from the teachers involved. The revision of instruc- 
tions aimed at increasing the^cla^ity and' siilipli city of 
expression; this revision was also/performed in co-ORera- 
tion with the teaqhers involved in the pilot study. 

In oMer to reduce'as much as possible non-desirable Mn- > 
structor effects' it was decided that pill instructions to 
the students as-ib, how they .should work with the program 
under the different e^jjerimental conditions, should be re^ 
corded on tape and played back via tape recorder , during t\\e 
introductory lesson (lesson.no. 3 in the schedule). By 
meafts of brief written instructions, handed out to the stu- 
dents during the PI lessons (lessons no. 4-9) - so called 
reminder sheets - we tried fuVther to diminish the need for 
oral 'instructions from the |5art^of the teachers in the ex- 
perimental groups. 

The Revision of the PUSL test is further described in sec- 
tion- 11 below. e!i ^ ' 

An excerpt "from the final version of t'he program "The main 
pat^ts of the simple sentence" is found-in a p p^ e n d i x 
4. ^ . - . • 
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7 The PUSL Test 



The sentence analysis tes*t used in the pilot atudy contained 
five subtests with in a^ll §0 itjams. Xhis t^st was revised on 
the basis of pilot study fjandings. Since the test was inten- 
. ded to make comparisons between studentsMn different grades 
poss.ible, it was' desirable that it permit a relatively great 
variation of scorefe above'as-well as below the expected to- 
tal mean, i.e. the mean-p value for all items, should be clo- 
se to So per cent. 

The experimental conditipns'studi^d here probably havS only 
marginal effects on T,earning, i.'p. these effects will be 
small compared to, the effect of 'the program as such. It wap 
t{iei=efore important that 'the test should differentiate maxi- 
mally .ih the interval around the total mean for all^roupsr 
which means that the majority of the test items should have 
p values fairly close to the average, 50 per cent (cf. A02: 
221f). ' , , : . 

The revised version of the PUSL test contains three sub- 
tests with 15 examples (sentences) eactr. In subtests A and 

each example comprises one test item, whereas in subtest ' 
C it comprises ^wo Items, whioh gives in all 60 items. This 
version of the 'test was given to 57 students in grade 6, who 




samplf- 

figure 11 .J ; The^mean p value is 52.6 per cent and 
47 out of eojtems have p values between. 40 and 69 .p^r cent. 

The test reliability, according to the Kuder-Richardson for- 
mula 20, cQmputed from data for this sample of 57 students 
wqs 0.98. 0 , ^ 

The PUSL test in this version obviously had the desired cha- 
racteristics as regards the distribution of item p values. 
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fim^re ll.l The PUSL te'stf distniktion of item p-val 
^ (no. of itenis = 60; no. of subjects = 57) 



*And the reliability coefficient of 0.98 poirfted.to a very 
high item homogeneity. The test could, then, be considered 
well suited for its purpose: to be used as criterion test 

'in the main study. One minor change was, however, made in 
the^'final version; the items were re-arranged and ranked ac- 
cording to p valuk. The testing time limit, was intended to 
be purely administrative, but in case some students still 
would not have time to cpmplete all test items, it was con- 
sidered wis^ to order them in this way. 

^ The reliability of the Yina'l version of the PUSL test was. 

* computed from data ,for the sample used in the main Study. 

* The following coefficients were obtained. 



Homogeneity 
Retest reliabiLit: 



KR20 



= 0.96 (posttest 1) 

= 0.79 (posttests 1 & 2). 



The final version of t'lie PUSL test Is shown, in a p p e n 
d i x 1 . ' ^ _ ' 

Instructions a;id time fimits for the final version of the 
PUSL test were determined on< the basis of p'rlot study ex- 
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^perlences and pf the try-out of the revised version. The 
three subtests were timed indiv1duafl]y: six minutes for 
each of subtests A and B, and ten minutes for subtest C. 
The instructions were identical on. all three test occa- 

^ sions (pretesting, posttesting* ] & 2), except for the in- 
troduction telling the students about the purpose of -the' 
testing etc.^The complete instructions are4iven-in ' 
a p p e n d i X 2 . ■ ^ 
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.The Students Participating in t*he Main Study 



12.1 ^ELECTION OF EXPERIMENTAL CLASSES 
♦ ^ ^ ' 

it was decided that the student sample should comprise 6 " ^ 
classes from each of the three grades 4.; ,5, and B> making^ a„ 
"total of 18 class'es. Random selection o-f classes or schools 
was not possible^ sinfce participation in the experiment was 
voluntary as far'^as th'e 'teachers were concerned. 

All teachers- in gradfes |4 - 6 in'^fbur schools in' Link'dping 
and two in Norrkoping were invited to participate. In three 
schools, two in. (Norrkoping ^d'one in Linkcping, all teachers 
invited agreed tb. oarticipate. These schools had in'all 20 
classes in grades 4 - 6,. and, 18 of them were selected as ex- 
perimental o^lasses.'The total number of students in these . _ 
classes was 464. . ^ ^ . , • 



12.2 THE STUCJeNTS' PRIOR KNOWLEDGE OF SWEDISHi GRAMMAR' 

• ■ ■% ' ' ' 

All experimental class teachers declared that t|ieir, students 
had had no systematic instruction on "Sentence parts 6r sen- 
tence analysis^^ri or to this study. The concepts 'subject' ^ 

■ ancf- 'predicate' had be&n mentioned by ithe teachers jin all «i 
grade 6 classes artd in some grade 5 cUsses - main'y in- con- 

' nection with the -teaching of English. The parts of speech 
'substantivel and 'v.erb^ had, aecordingl to the teachers, 
been treated rather thoroughly in Sill classes in grades 5 
and 6, and to soine extent in grade 4. Pronouns had been trea- 
ted ^cursorily in a few classes, mainly ih, grade 6.K . 
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The student* population of the present study was' defined so 

ClS to Include only normal class students in'grades 4 6 . ^. ^ . 

who answered- correctly less than half .of the 'PUSL test iterns '* . ■ » 

at the pretesting. Students with a pretest scor$ of 30 or* 

more w^re^ consequently, excluded from the main^ study. Stu-' 

dents who were absent from the pretesting were also exclu- • ^ . ' 

ded . . I .i . " ^ 

.T a b 1 e 12 . 1 shows -how many students participated in 
the pretesting and how many score'd above 30 points. The per=^' 
centage of students with pretest* scores of .30 or more was 
0 In^grade 4, 2 per cent in grade, 5, and 8 per cent in grade ^ ^ 
6. Pretest means (the variable PRE) foe the three grades , 
were as^ follows (students jWit'K 30 points .or more excluded), , 

• Grade 4: 3.1 . , • 

4 ., Grade 5: - 4.6 

• Grade 6:11.3. . > 

Obviously, prior knowledge of grammar increases with in- 
creasing gradf le</el. Still, the grade 6 mean is sufficient- v 
ly Jow for le?rn.i'ng effects,, fol J owing the PI to be measured . ' ^ 
by means of t|[ie PUSL tes1> even in this grade (the maximum 
possible score is 60). ^ 



12,3 ALLOCATION OF STUDENTS TO' EXPERIMENTAL GROUPS 



Within each experimerital class the students were assigned / ^ 

by lot to six different groups. Boys and girls were allotted ^ 
to groups separately, so as to obtain a fairly even sex / 
distribution in each experimental condition (RF combination)./ 

In Order to avoid delays between th^ pretesting and the ex- 
perimental lessons the allocation of. subjects to 'groups was 
Derformed prior to the analysis of ^Vetest data. Students o 
whO'Were absent from the pretesting or who, due to high, pre- • 
test score, did not belong to the student population, were 
thus a+so-^lotted to experimental groups. The distribution • 
on experimental groups of the 41 9 'students who tookVrt in 

- the pretesting and belonged to the^population was, however, 

^ relativejly eVen, as can be seen in t a bM e 12 /2 . 



!rabte''l2.l' Number c^f students in eacti class wttti pretest 
score below or not below 30 points. 



Boys Girls Total 



•Grade 


Class 


PRE 


PRE 


. 'PRE 


PRE 


' PRE 


PRE 




<30 


>30 


. <30 




<30 


>30 


4 




10' 




5 




15 






2 


' 9 




11 




20 






3 


.11 




10 


- 


21 


- 




4 


17. 




^ 10' 




27 






5 


■ 13 


■ - 


r 14 




27 






. 6 






19 


- 


27 






Sum 


68 




69 • 




137 




5 ] 


1 


7 


1 


11 




18 


2 




2 • 


11 s 








20 








i4 




15 


- 


29 






4 


14 




-13 




* 27 






5 


13 




15 




28 






6 


14 


1 


- '14 




28 


1 




Sum 


73 - 


2 


77 


1 


" 150 


3 


6 


^ 1 


12 


1 




1 


21 


2 




2 


TO 






1 


22 


1 




3 ' 


14 




, 11 


1 


25 


1 




4 


6 


2 


12 




18 


2 




5 


11 


2 


11 


2 ' 


22 


4 




6 


. 13 


1 


11 


1 


24 


2 




Sum 


66 


6' 


66 


6 


132 


12 


4-6, 


Total 


207 


8 


212 


7 


419 


1^ 



\ 

' . .•• ■ 107 . 



' ' ' 
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Distribution of students on the 36 cells^ — ' 
in the design R-xFxGxS (N = 419). 


• 


Grade / Sex 


F1 


P.5 


F1 


R2 
F.5 


Fo : ; J , 




64 M . 

F ' 


12 
11 


n- ^1 

12 12 


11 
12 


11 
12 


CVJ o 




G5 - M 


• Tl 


13 10 


14 


12 


_13 


— . 


,:. . -.p-- 




13 14 


11 


12 


13 




66 M 

F • 


n 


iT" 11 

11 10 


11 


•11. 

1 1 


in . 
11 




Sum 


71 


71 68 


70 


69 


70 








y 










\ / 




\ 

1 ■ 
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Experimental Conditions in the Main Study 



The different values of the design variables response re- 
quirement (R) and key answer frequency (F) have been defihed 
in general terms (section 5.4 above). The experimental con- 
ditions in this study are the six possible pairs^of R and F 
values: . i 

RlFl , R1F.5 , R1;F0 , RZFl , R2F.5 , R2F0. « 

Each condition is operationally defined by the combination 
of program version and- .instruction presented to the students 
in each experimental group. The program varies for different 
F values only (different amount of key answers), whereas 
the instructions to the students vary both with R and with 
F. • 

Here the experimental conditions are described fay means of 
quotations from the taped instruction presented during the 
introductory lesson (no. 3), and from the 'reminder sheet' 
•handed out to the students at the beginning of each PI les- 
son (nos. 4-9). ^Formulations which are specific to a par- 
ticular condition are emphasized. 

Rl (overt response requirement), for groups 1 - 3 

.From instruction on tape: , : j 

!'In some frames there is a question and an answer space. 
•^There^you shall write t h e a n s w e r t o 
t h^e q u e s t i 0 n . In other frames there is some- 
thing inissing.. There you shall f.i 11 1 n t h e 

m i"s s 4~n-^g w ord o r words. _. If you 

discover that your answer is wrong, you are free to go 
back and correct it."' * 

^Trom 'reminder sheet': 

"W r, i t e answers to all questions and fill 
i n all missing wordsl.'^ ^ 



R2 (covert response requirement), for groups 4-6 

From instruction on tape: 

"In some frames there is a question and an answer '^pace. " 
' There you shall think outthe answer 
/but not write it down. In other frames 
there is something missing. There you shall think 
out tiie missing word or words 
' b u t n 0 t f i 1 1 i t i n 

From 'reminder sheet*: ? ■ f 

"Think a u t. a n s w e r s to all questions and 
t h i n k 0 u t the missing words!" 

Fl (100 per'gent key answers), for groups 1 & 4 

From instruction on tape: 

"When you have finished a. frame, you shall check that 
you have written the right answer 
(Rl)/ t h. 0 u g h t out the r i g h t a n - 
s w e r (R2). The right answer - or the missing words 
- are written in the left hand margin on the following 
page. 

Maybe you think that you can look at the right answer 
first and then c o.py i t (Rl)/ with - 
out first t h i n k i^ n g out your 
own a n s w=e r (R2). Of course you can, but you 
won't learn much. - If you want to learn this thorough- 
ly you must first think yourself (R2)/ 
think and answer yourself (Rl), 
and then check that' you have written (Rl)/ 
thought out (R2) the right answer." 




Frdh 'reminder sheet': 

"Read the right answer and check that you have w r i t - 
.ten (Rl)/ t hf o^u g h t out (R2) the. right 'an- 
swer before y,Qu go on to the next frame!" 

F.5 (50 per cent key answers), for groups 2 & 5 

2 

From instruction on -tape: 

"Many times you'll find the right answer in the left hand 
margin on the following page. Then you shall check that 
you have written the right answer 



'{Rl)7 thought out th.e right an- 
swer (R2). Sometimes the right answer is not there. 
Then you cannot check what you've w r i t t e n (Rl)/ 
thought ou tJR2). You'll surely get along fine 
with these questions too, if you just think carefully. 
- But remember*^ always to look for the right answer, 

■ you'll often find it, ' 

From 'reminder sheet': 

• "Always look for the right answerl - If 1t is there, 
check that you've written (Rl)/ thought 
out (R2) the right answer, before you go on with the 
next frame:" , r ^ 

• FO (zQro per cent key answers), for groups 3 & 6 

From instruction on '-Eape: 

"When you have- answered" a question, you would perhaps 
like to know if your^-answer was correct. But the 
rightanswers are not written 
in t. h e booklet. If you have read the frame 
; and thought about the answer carefully, you'll surely 
know what is the right answer." ^ 

From 'reminde? sheet' : 

The instructions for the different experimental conditions > 
are, then, paired combinations of the above R and F instruct , 
tions. Other instructions are the same for all groups. The 
entire instructioji given to group 1 (overt response require- 
ment and 100 per cent key answers) is found in a p p e n - 
d i X 3 . , . ' • 
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Carrying Through of the Main sftudy 



s 



14,1 



LESSON PLAN AND GROUPING OF STUDENTS 



The lesson plan of the main study was the same as that' of 
the pilot study (section 9,4 above). During testing lessons 
(nos. 1, 2, 10, and U) the students all worked in their 
own classrooms. Testers were the. experimental clasps teach- 
ers, forming pairs and interchanging classes within each 
pair. During the introductory lesson (no. 3) the students ; 
were regrouped so that only students working under the same 
experimental condition worked in the sarfle room* During the 
PI lessons (nos. 4-9) the students were grouped according 
to response requirement, i.e. Rl groupl^from two classes 
worked in one room led by one teacher; and R2 groups fro% 
these classes in another room Ted by the othep teacher. 



It was considered important that students working under dif- 
ferent response requirements should not influence each * 
other. Rl demanded overt responding and should take longer 
t^me than R2 (covert responding). Rl students working to- 
gether with R2 students might, therefore, be tempted to 
work too rapidly when they saw other students finishing 
their booklets very quickly. / 



During testing lessons, on the other hand, it was important 
that th*fe situation should.be^ as far as possible, the same 
for all students regardless of what experimental condition* 
they had worked under. 
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14.2 INSTRUCTIONS TO^ESTERS/ INSTRUCTORS 



The teachers who functioned as testers or instructors^^in ^ 
the different experimental grduf)s were given detailed writ- 
ten instructions about wKat to da and say. At a meeting 
prior to the study they were informed about the design of 
the study and the purpose of the different in$tructions. 
The importance of following*^ in every detail, all instruc- 
tions was strongly emphasized. 

A sample of written instruT:tions ;to testers/instructors is 
given in a p j) e n d i x. 2 which contains instructions 
for • • ^ . ^ . 

lesson 2'(pretest1ng lesson); ' ' 

lesson 3 (introdyctory 'lesson) ; ''and 

lessons 4 - 9 (PI lessons). ' * 



14. a INSTRUCTIONS TO THE STUDENTS' 



During testing lessons- (nos. 1, 2, 10, and 11) the students 
were instructed orally by 'the testers. When students were 
introduced to the experimental conditions (lesson no. 3|, 
a brief instruction was given orally by the instructor (see 
appendix 2). The major, part of the'lnstructfon was played 
back on a tape recorder. The differences between experi- 
mental conditions as regards the introductory lessorr have \ • 
been described, in chapter 13 above. The full instruction 
for one of the experimental groups (no. 1: RIFIJ is given 
in appendix 3. • * ^ ' 

Dur*ing the PI lessons (nos. 4 - 9) the students worked in- 
dividually after a brief oral 'starting instruction' (cf . 
appendix 2). Each student wa^ also given a 'reminder sheet' 
Where the working instructions were summarized. Ther^ was 
one 'reminder sheet^ for each experimental condition (com- 
bination of response requirement '.and key answer frequency). 

As in the pilot study, -students who had finished their pro- 
gram booklets stai^ted workl^ig on arithmetic exercises. The 
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aim of this was to avoid disturbances and irritation among 
students who^, having finished the booklet, would otherwise 
have nothing to do. A PI lesson wKs concluded when all stu- 
dents In the room had finished their booklets. 
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Treatment of Main Study Data 



15,1 



THE RAW DATA SET 



A complete set of raw data for one subject should comprise 
values in all the variables listed in table 15,1. 
The data analysis in^the main study is. based solely on com-, 
plete individual data setj;, which presupppses that the in- 
dividual has been present iiuring all eleven experimental 
lessons. Complete data sets are available for 3T2 subjects, 
distributed on the cells on the R x F x G x S design as ^ 
shown in table 1 5.. 2 . The total set of raw data 
in this study consists, thus, of 31?. individual raw data 
sets as described in table 15.1. 



15.2 MISSING DATA 



In all 419 students took part^ln the pretesting and had 
less than 30 points on the PUSL test (cf. table 12.1), and 
complete data sets are - as we have seen t^/aTSTlable for 
312 studeni>s. The proport^n of students with missing data 
is,''then, about 25 per cent'. This %ay seem .a rather high 
figure, but i:t. should be noted that it includes all stud^ents 
who were absenf'from any one out of eleven lessons during a 
period*'of several weeks. The absence was in all likeliness 
due almost entirely to' normal causes, mainly temporary ill- . 
ness. . 

Pretest (PRE) 'and verbal intelligence (D2) scores are avail- 
aMe for the 107 students with missing data. The mpans for 
thk group have been compared with the corresponding means 
for the remaining 312 students, and the results are'summa- 
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TablM iSit Raw dati^ set In the main stucly, 



Variable Deflnttion' 



POST! 
P0ST2 
PIT 

' PRE 
D2 

D3 " 
W 



CW 

G 
R 

F 
S 
C 



PUSL-test score at posttesting no. 1 
Ditto' at posttesting no. 2 (8 weeffs later) 
Total gross working time spent on program booklets 
1 - 6, 1n. minutes 
PUSL-test score at pretesting 
0BA:2 score (verbal intelligence) 
D3A:3 score (inductive, intelligence) 
Total number of program 'responses that are omitted, 
erroneous or originally err^oneous but corrected by 
the student ■ , 

Total number of prx)gram responses that are origi- 
nally erroneous but corrected by the student 
Graj^ level 

Experimental condition in terms of response requi- 
rement 

Ditto la terms of key answer frequency 
'Sex of student 

No. of class to which student belongs 



Tctble 15.2 Stjdents with complete set of raw data: distri- 
bution on cells in the design R x F x G x S 



G 


S 


Fl 


F.5 


FO 


Fl 


•R2 
F.5 


FO 


Sum 


4 


M 


9 


9 


9 


* 6 


6 


10 


49 




F 


8 


8 


6 


11 


10 






5 


M 


8 


10 


9 


10 


7 


10 ■ 


54 




F 


10 


n 


9 


10 


9. 


12 


^1 


6 


H 


10 


7 


8 


9 


8 


10 






F 


8 


8 






10 


6 


46 


Sum 




53 


53 


47 


54 


50 


55 


3^2 



rlzed In table 15,3^ None of the, computed t- 
values for the mean difference in thi^ table is significant, 
and most of them are close to zero. Apparently, students 
liminated because of missing data do not differ markedly 
from the remaining sample as regard^ pretest (PRE) and ver- 
bal i nte'l 1 igehce ( D2) scores . 

The distributions of missing data oa- experirrlental gjVoups 
have also been studieci. The observed frequencies are compa- 

, red* with the frequencies 'expected uVid^r the cass.umption of 
equal frequencies of students with missing data (i.e. 25.5 
percent) in all subgroups. Such analyses have been perfor- 
med separately for *he variables sex, c^a'de level, and ex- 
perimental condition. The results are summarized in 
table 15.4. All three chi^-values in that table 

' are very far from significant, which indicates that the 
variation between groups as to frequency of misSing data is 
not greater than normal chance variation. The distribution 
of missing data can, consequently, be assumed, to be inde- 
pendent of the design variables. R, F; G, and S, 

To sum up this discussion, it can.be observed that the fre- 
quency of students with missing data may seem large, mainly 
due to a very strict :<Iemand for completeness of the. data 
sets, but that there is no indication' that:^the absence of 
these V.data has affected the differences between experimen- 
tal groups in the various ciriterion variables. 



15.3 VARIABLES GENERATED FROM RAW DATA 



On the basis of the raw data (cf. table 15.1) the following' 
variable values were computed and regis'tered for each stu- 
dent. * . 

SD2, SD3:\ DBA:2 and DBA:3 score respectively, expressed 
\.in a stanine scale for each grade (i.e. with 
• * ' eaciiu^rade mean equalling 5). 

PREM(C): Mean pretest' score on the PUSL test for thf 

student's class' (C), computed from data for ^ 
\ all students participating in the, pretest. 



Tcdile 35. 3* Compartson betv^een elimtnated and included sub 
jects vri.th regard to PRE and D2 means. 



Vari- 


Grade 


Elimin. 


Ss 


Incl. : 


3s : 




able 




n 




s 


n 


M 


e 


U 1 1 1 


PRE 


. 4 


„ 38 


3.3 


2.6 


99 


3.1 


2.6^-. 


+0.4 




5 


35 


4.8 


5.3 


115 


4.6 


6.0 


■t;0.2 




6 


34 


10.1 


9.1. 


' 98 


)1.3 


8.6 


-0.7 




4-6 


' 107 


6,f) 


6.7 


312 


'6.2 




-0.4. 


02 


4 ' 


38 


21.9 


5.7 


99' 


20.6 


5.8 


+1.1 




5 


35 


25.5 


7.3 


115 


25.6 


5.2 


-0.1 




6 


. 34 


■27.9 


6.1 


98 


28.2 


6.2 


-0.3 




4-&' 


107 


25.0 


6.8 


312 


24.8 


6.5 


+0.2 



Table 16.4 Distribution of subjects with missing data. 

E = observed no. of eliminated Ss 
Eg = expected no. of eliminated Ss 

' (a) Sex * (b) Grade level 



S 


\ 




6 


^0 


■ ^e 


M 


54 


52.86 


4 


38 


34.98 


F 


53 


54.14 


5 


. 35 


38.31 


Sum 






6 


' 34 


33.71 


107 • 


107.00 












Sum 


' 107 


107.00 


chi^ = 


0.05 


, df^l , N.S. 














chi^ 


= 0.55 


, df=2 , N.S. 



(c) Experimental condition 



Cond. 


^0 




Cond. 




^e 


RlFl 


18 


18.13 


R2F1 


16 


17.88 


R1F.5 


' 18 


18.36 


R2F.5 


19 


17.62 


R1F0 


21 


17.36 


R2F0 


15 


17.88 


Sum 


57 


53.62 ^ 


s^,^^^Sum 


50 


53.38 
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PREM(GC): Mean of class means (PREH(C)) for grade 6. 

PREM{G): Mean pretest score for grade G, computed .from 
data for students included in the results 
analysis, ' , 

On the basis of raw data and the data described .here values 
'for the 312 remaining subjects In the var-^ables to be used 
in result analyses were generated a^ described below. 



15.3.1 Pre-treatment knowledge 

The variation in pre-tteatment knowledge (measured by pre- 
test score^ PRE) can" be regarded as having three components: 

(1) an individual component (variation between Individu- 
als- 1n, each class); > ' ^ - 

(2) a clas^ component (variation between classes in each 
grade) i and 

(3) a gradfe component (variation between grades). 

It is conceivable that each qf these components affect test 
performance in a study of the present kind. 

The grade' component Is already accounted for, since grade 
level (G) is a design variable in the study. The class com- 
ponent cbuld be accounted 'for In the same manner; by inclu- 
ding class membership as a design variable. This would, how- 
ever, have led to almost Insurmountable missing data* prob- 
lems In this case, so another^method of accounting for the. 
class component In pretreatment knowledgie ^lad to be devised. 

The class component of the variable PRE reflects, among 
other things, differences between classes as to the nature 
and amount of instruction received on the subject matter 
covered by the test. These differences are likely to affect 
the students' capability of learning the tasks set by the 
instructional program. If, for example, two students oelong 
to classes with different pretest means due to different 
amounts of teaching. It Is conceivable that these students 
ability to learn from the. program differs, even n they have 
the same pretest score. Instead of simply letting the vari- 
able PRE he included In the analyses tm new variables are. 
therefore, generated: 
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(1) The ^indivi dual 's dep'ation -from his class mean ^ 

PREI = PRE - PREM(C) 

(2) The daviatiOR of the class mean from the mean of 

^ ^ . class means for the.grade^ to which the class belongs 

; * ' PREC = PREM(C) - PREM(eC) 

Class means are computed from scores for all students parti- 
cipating in the pretesting, including those who scored 30 
points or more and do not belong to the population. This 
mean is the one-most appropriate for the present purpose, 
since it reflects 'the level of .knowledge in the class cTs a 
whole. For the reason the grade mean used in the abo^e 
formula (2) is the mean of class means* which gives each 
class eqfual weight regardless 'of the number of students in 
it. All class means and gra^e means are given in t a b 1 -e 
15,5, 



Tdbleh^.^ Class means*, PREM(C), and grade meins» 
PREM(GC) in the variable PRE. 



Class G 4 G 5 G 6 



•ihO 10.0 

3.8 15.8 

5.8 15.2 

4;6 15.7 

7.4 \ZA 

6.0 11.5 



Mean 3.0 6.4 13.6 



. Note: Class numbers are arbitrary serial 

numbers.. There is no pcirticular rela- 
tionship between classes in^ different 
grades with the same number. 



In analyses where pretest result enters as a covariate the 
variable PRE-ii, thus, replaced by the two variables PREI 
(individual component), and PREC (class component) . The grade 



2:6 

3,2 

5.5 
3.2 
3.0 
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component is a design variable and therefore not used as a 
covariate:, it enters, however, into the computation of per- 
formance increment from pretest to posttest (sectionTb.a.b 
below). 



.15,3.2 Intelligence measures 



Intel l.t^ence scores, as measured by the tests DBA: 2 and 
DBA: 3, were intended to be usjed as covariates in the re- 
sult analyses. The variation in intelligence scores con- 
tains, however, also a grade component. Since grade is a 
design variable in this study, it was desirable to elimtoa-^ — 
te the grade* component from the covariates. In order to ob- 
tain this the raw scores D2 and ;D3 have been replaced by 
SD2 and SD3, i.e. the;corresponding stanine scores, compu- 
ted for each grade separately. Since the means of these sta- > 
nine scales are 5 for all three- grades, the grade component 
in intelligence score variation is eliminated from the vari- 
ables SD2 and SD3. . ' - 



15.3.3 PI time measure 



As figure 1 '5 . 1 shows the variable PIT has a mar- 
kedly skewed distribution, which probably implies that the 
means and the standard deviatiohs for subgroups covary. As 
suggested by Edwards (76:13Q), it has been attempted to eli- 
minate the skewness by means of a logarithm transformation, 
viz. LPT = log PIT, 



The variable LPT is, as f i g u 
ximately normally distributed. 



re 15.2 shows, appro- 



15.3.4 Program response variables 

For students in the Rl experimental groups, who wrote answers 
in their program booklets, the raw data set contains two re- 
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sponse error variables: number of incorrect, original ly in- 
' correct but then corrected, or omitted answers in tne book- 
lets 1 - 6 (the variable W); number of originally incorrect 
answers that were spontaneously corrected by the student 
(the variable CW}. From these two v^iriables have been com- 
puted. 

UCW = W - CW , t.e. number of uncorrected errors and 
omissions; 

PCW = CM/W , i.e. proportion of errors corrected by the 
students. 

The distributions of all four error variables (W, CW, UCW, 
and PCW) deviate from the normal distribution, .as can be 
.se^n in f i g u r e 1 5 . 3 . 

In order to normalize the distributions the variables W, CW, 
and UCW were transformed according to the formula 

x' = log{x +1) 

to LW, LCW, and LUW respectively (cf, 76:130). The variable 
PCW, which is a proportion, transformed as suggested by 
Edwards (76:131) to ■ » - 

APC = arcsin(PCW)^^^ ^ 

The dtstnibutions of the new variables LW, LCW, LUW, and 
APC are shown in figure 15.4. The three first- 
mentionerf^variables have approximately normal distributions, 
while the distribution of APC is more rectangular. Compared 
to the raw score distribution (PCW, figure 15.3d) APC is', 
however, much more symmetrically distributed, and it must 
therefore be considered more appropriate than the raw score 
for use in variance analyses. 



15.3.5 Learning effect measures 



The analysis of pilot study data indicated that learrffng 
effect in a study of this kind is best 'investigated by means 
of a covariance analysis with posttest score -as variate, and 
pretest score and verbal intelligence score as covariates 
(cf. section 9.7.1 above). In the main study pretest score 
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?iauve 15.1 The variable PIT: distribution of scores 
(N = 312). 
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Figure 15.2 The variable LPT (= logPIT): distribution 
of scores (N = 312). 
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is replaced 6y two prestestscore components: ^indlYidual and 
class component (section. 15.3.1 above). 

Since the gain from pretest to posttestMS of interest here, 
and not the posttest performance level in absolute tentis, 
posttest scores are replaced by the difference between post- 
test score and pretest mean for the grade to which the stu- 
dent belongs, viz. 

EDI =.P0ST1 - PREM(G) ; and 

ED2 = POST? - PREH(G.). 

The variables EDI and ED2 are not measures of learninQ ef- 
fect, since' the individual and class components of the pre- 
test score are not accounted for. Learning effect measures 
are obtained by adjusting EDI -and ED2 for the regression on 
the covarlates PREI (indlvtdual pretest score component), 
PREC (class pretest score component), SD2, and SD'j. 

Immediate learning effect (El) isj then, measured by the 
following variable 

El = EDI - Eb^(x - X) , where x = PREI, PREC, SD2, SD3.^ . 

and where bx is the partial regression coefficient within 
cells for covariate x. And retained effect ^retention) is 
measured by 

E2 = ED2 - T.h U ^ x). 



15.4 - SUf4MARY OF VARIABLES INCLUDED IN THE RESULT 
ANALYSES 



In the analyses of data from the main study the following 
variables are, then, employed.* ^ 

' Vftrlates 

EDI = POST! - PREM(G) Diffcrfince between individual post- 
test 1 score and pretest grade mean. 

ED2 = PQST2 - PREM(G) Ditto for posttest 2. 



An 



LPT = logPIT The logarithm for the Individual PI 

time value. 

El E2 EDI or ED2 adjusted for regression 

on PRE], PREC, SD2, SD3.' 

Covariates for all groups 

PREI = PRE - PREM(C) - Difference between individual pre- 
test score and class means. 

PREC = PRE - PREM(GC) Difference between class and grade 

pretest means, 

SD2 = stanine{D2) Individual 02 score expressed In 

stanines, within grade. 

SD3 = stanine{D3) Ditto for D3. 

LPT See above under variatesl 

Covariates for Rl groups only 

LW = logtW + 1) The logarithm for the number of. 

errors and omissions in program 
answers (booklets 1-6). 

LCH = 1og(CW + 1 ) Ditto for corrected errors. 

LUW = lQg(UCW + 1) Ditto for uncorrected errors and 

omissions. ^ 

APC = arcsin{CW/W)^^^ The proportion of errors corrected 

by the student, transformed through 
arcsin transformation. 



15.5 METHODS OF COMPUTATION AND ANALYSIS - 

The raw data set (section 15.1 above) and the values SD2, 
SD3, PREM(C), PREM(G), PREM(GC) for each subject (N =312) 
were punched on punch cards. The variable transformations 
described above (section 15.3) were performed by .computer 
in connection with the use of standard statistical analysis 
programs from the so called BMD series (cf. 75).. 



The covariance analyses were performed by means of a combi- 
nation of automatic and manual computations. Partial re- 
gression coefficients, adjusted within cell variances, ad- 
justed cell means etc, -for the design R x F x G x S and for 
different sets of vartates and covariates wer-a obtaified in 
output from the computer program BMD04V, which employs, stan- 
dard statistical formulae (cf, 75), On the basis of such 
output variance analyses of adjusted cell means were per- 
formed manually using the methodv described in the report on 
the pilot stud^ (section 9,6 abovfe^L In some cases the va- 
riation between groups has been analysed into a linear and 
a quadratic component employing fprmulae given by Edwards 
(76:«hapter 10"). Multiple regression coefficients have in 
certain cases been computed manually on the basis of zero- 
order correlation coefficient matrices; in those. cases the 
JDoolittle method has been emoloyed (124:326ff},,M 



ATI machine computations were performed on a 6E 625/635 
computer owned by Industridata AB, Solna. 
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Data Analysis Results 



The word 'significant' in the discussion of main study re- 
su.lts means throughout 'statistically significant at theiS 
per cent level (a =^0.5)', unless sotne other meaning is spe- 
,cifically stated. Test variable values are marked with one 
asterisk ('»^)', -if they exceed the limit for sigrifficance at^ 
the 5 per cent level, and with two asterisks if they 

exceed the limit for^ significance at the,l p^r cent level. 



r 



16.1 LEARNING EFFECT 



Immediate learning effect is measured by El, i.e. the EDI 
score (POSTl - grade pretest mean) adjusted for regression 
within cells on the covariates PREI, PREC, SD2, and^SD3 (cf. 
sections '15.3.5 and 15.4 above). The observed coefficients , 
of partial regression within cells (b) and their correspon- 
ding t-values for testing the null hypothesis (b =>P) are 
as follows. 

^ PREI PREC^ ' SD2 SD3 

b 0.48 0.22 3.27 1.26 

t^ 4.0^ 0.6 8.0*^ 3.1**** 
. D ^ _: : 

Obviously, posttest 1 results are primarily correlated with 
intelligence ,(SD2, SD3) and the individual component of the 
pretest score (PREI), but not with the general level of pre- 
treatment knowledge in the class (PREC). 

<> 

All cell means in the variable El in the design R x F x G 
X S are given in table 16.1. 
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Tabl& 16.1 Learning effect (El): 'cell means in the design. 
R X F X G X S. ' 





G S 


Fl 


Rl 
F.5 


■ FO 


Fl 


DO 

F.5 


FO 




4 • M 
F 


9.72 
12.78 


13.82 
11.01 


13.60 
.14.71 . 


9.18 
' 9.72 


2.67 
5.77 


8.36 
9.55 




5'- M 
F 


< 7.04 
14.37- 


11.31 
15.49 


14.61 
23.53 


15.96 
8.92 


19.25 
10.93 


11.28 
5.37. 




6 M 
F 


'15.21 
17.81 


15.31 
13.69 


22.93 


15.22 
16.02 


13.05 
21.51 


17.11 
19.62 




4-6 M,F 


12.82 


13.44 


17.97 


12.50 


12.20 


11.88 



16.1.1 Variance analysis of the design R x F x G x S 

The results of the variance analysis of the El cell means 
(table 16.1) are presented in table 16.2. The 
Interactions R x F and R x G have been divided into a li- 
near and a quadratic component and t-values for these com- 
ponents are also included in table 16.2. 

On the whole, the variation in experimental conditions has 
not led to any marked variationsMn learning effect. Except 
for the significant R x G x S interaction, which is very dif- 
ficult'to interpret, there are no significant F-quotients / 
involving R or, F. ^ ' 

According to the hypotheses formulated in this study (section 
7.1 above) some sources of variation are of particular inte-/ 
rest, viz. R, R x F, and R x G. / 

! 2 ° S ?• ® n"" ^ " ^ ^ " t ( R ) The F-quo- 
tlen.t for R is close to the significance limit (3.67 as a- 
gainst 3.88). The El mean for all Rl cells is 14.74 as com- 
pared to^2.19 for all R2 cells. This result is compatible 
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Table 16.2 


Learning effect (El): variance analysis of cell , 
means tn the destgn R x F x S x S, 






Source of 
variation 


df • 




F 




• 


> 


R 

F . 

G 
S 


1 

2 

- 2 

1 


58.47 
19.17 
151 .38 
14.18 

I 


3.67 
9.50 


3.88 
3.04 
3.a4 
3.88 






R X F 
R X G 
R X S . 
F X 6 
F X S 
G X S * 


2 
2 
1 
4 
2 
2^ 


28.80 
18.21 
28.34 ^ 

8.15 
. 1.82 

6.93 


1.81 • 

1.14 

1.78 

<1 

<1 

<1 . 


3.04 
3.04 
3.88 
2.41 
3.04 
3.04 






R X ? X G 
R X F X S 
R X G X S 
F X G X S 


' 4 
2 

* 2 
4 


20.04 
12:57 f2 
60.44 
1.59 


1.26 

3.79'* 

■ <1 


2.41 ^ 

3.04 

3.04 

2.41 






RxFxGxS- 4 


2.21 


<1 


2.41 






Error (within 
cells) "^'"^ 


15.94 








- 


Linear and quad^atip 


aomponente of certain interaationB : 






Inter- 
action 


Component 


d 




^d 






R X F 


Linear 
Quadratic 


-34.61 
23.53 


19.54 
33.91 


-1.8 - 
,+0.7 






R X G 


Linear 
Quadratic 


29.54 
1.96 


20.10 
m?.93 


+1.5 
+0.1 


• 
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with the hypothesis that the learning effect is not much 
greater under overt response requirement (Rl) than under 
covert (R2). 

Interaction between response re- 
• quirement and^key answer frequen- 
ce y ( R X F ) : The interaction R x F is primarily linear, 
with a small quadratic omponent (table 16.2). The t-value 
for the linear component is fairly close to the significan- 
ce limit (t = -1.8). The interaction is graphically illust- 
rated in' figure 16.1 , which shows that decrea- 
sing key answer frequency (F) is coupled ,with increasing 
learning effect when overt responses are required (Rl) but 
not when covert responses are required (R2). The trend of^ 
the result is, thus, in agreement with the hypothesis about 
differential effect of key answer frequency under dif.ferent 
-res^nse requirements (cf. chapter 7). 

Interaction between response re- 
quirement and grade- level (RxG): 
This interaction, too, is mainly linefir (table 16.2); The 
nature of the interaction is seen in figure 1 6 , 2 , , 
which showis'that the^difference in terms of effect (El) bet- 
ween Rl and R2 groups decreases wtth increasing grade level 
(G 4, 5, and 6). In grade 6 there is practically no diffe- 
rence between Rl and R2* groups. Although the test variable 
is not significant, this result gives some support to the 
^ hypothesis that, overt response requirement (Rl) is most im- 
portant in the lower grades and less important tn the higher 
ones (cf. table 7.1 abpye). 

The significant interaction R x G x S (table 16.2) implies, 
however, that the interaction between response requirement 
. and grade> level (R x 6) is different for boys and for girls 
(S), as illustrated in figure 16.3. For grades 

4 and 6 (on the basis of which the linear RxG component is 
computed) the outcome is roughly similar for both sexes, 

' whereas in grade 5 'there is a marked difference: for grade > 

5 boys Rl has led to less learning than R2, while the oppo- 
site is true for grade 5 girls. A sex difference of this , 
kind could not be foreseen on the basis of previous research, 
and a satisfactory explanation cannot bp presented here. 

0 
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16.1.2 Grade by grade variance analysis of the R F x S 
; design ! : 

a \ ■ ■ 

' " * 

The R X F X G X S can be regarded as three replicaterof a 
R X F X S design, one for each of grades 4, 5, and 6v{G). 
The results of a grade by grade analysis^ of t^e R x F x S 
design is surmiarized in table 16.3, 

^ iJ'^r^^ ^ F-quotient for R Is significant, 

while other F-quotients are far from significant. Thetl 
"^u^"u^?^ ^2.60 anrf for all R2 groups 7.54, 

Which 1mplies\ that overt response requirement (Rl)^in grade 
4 leads to conislderably higher learning effect than covert 
response requirWient (R2). Variations in key answer frequen- 
cy (F) seem to h^ve no or little inf.luence on learning at- 
this grade level.X . 

\ ■ r ' ' , 

For g r a d e^ 5 (G5) only the interactions R x F and 
R X S are signl\ficant. These ar6 illustrated in f i g u - 

^? ^ J ? • ^ " ^ ^ • ^ • The R X F interaction im- 
plies that a reduction of key answer frequency from 100 
(Fl)-to 50 (F.5) per cent has some positive effect regard- 
less of response requirement (R), whereas removal of all 
key answers (FO) has a still more positive effect when Overt 
responses are required (Rl), but a clearly negative effect ' 
when^covert answers are required (R2). The R x S interaction 
in grade 5 indicates that, the overt response requirement 
iRl) has a - compared to the covert response requirement 
(R2) - negative effect on boys, but a positive effect on 
girls; a finding difficult to explain. 

For grade 6 -{GiS) there are no significant F-quctients 
for variatio/i between groups (table lj5.3); in fact, all the 
F-quotients are quite small. It seems then as if variation 
as to response requirement (R) and key answer frequency (F) 
has very little influeijice on grade 6 students' learning from 
the program *The main parts of the simple sentence'. 

This grade by grade analysis of learn^no effect data is, in 
a sense, only another way of presenting the results already 
described of the overall analysis (section 16.1.1 above). 
. Still, the grade fay gfade analysis singles out particularly 
clearly a couple of Interesting findings: 



Table IS. 3 Learning effect (El): grade by grade variance 
analysts of tfie design R x r x 5- • 



Gra- Source, ^2 p 

de of var. ^ , » 

4 R ' V 76.96 .4.83* 3.88 

F 2 10.71 <1 3.04 

S, '1 3.19 ^ .<1 3.88 

R X F 2 ' 10.24 * <1 -3.04 

R X S 1 1.00 <1 3.88 

F'X S 2 0.70 <1 _ 3.04 

R X F X S 2 . 4.66 <1 ' 3.04 

5> R . 1 17.86 1.12 3.88 

F 2 " 7.97 <1 3,04 

■ S 1 . 0.06 <1 3.88 

■ V R X F 2 51.92 3.26'; 3.04' 
' . "r X S 1 144.91 . 9.09'" 3.88 

FxS • 2 3.26 '<! 3.04 

■ R X F X S 2 0.19 <1 3.04 

6 R 1 0.06 <1' 3.88 

. F- ' 2 16.79 '1.05 -3.04 

: S 1 24.80 1.56 3.88 



R X F . 2 6.91 <1 - 3.04 

R X S 1 3.30 <1 3.88 

FxS » 2 1 .04 <1 . 3\p4 

RxFxS 2 11.95 <1 "3.04 



4-6 Within 272 15.94 
■cells 



A, 
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- The overt response requirement (Rl) has led to signifi- 
cantly better learning effect than the covert response 
requirement only at the* lowest of the grade levels in- 
vestigated, grade 4. ^ * . 

• - A significant interaction between response requirement 
(R) and key answer frequency (F), in accordance with the 
hypothesis stated in chapter 7, has been demonstrated; 
only'for grade 5, that is true, but this is the grade 
for which the program employed in the study was origi- 
nally intended. 



16.2 LONG-TERM LEARNING EFFECY 



The long-tenn learning effect is measured by E2, i.e. the 
ED2 score (P0ST2 - grade pretest mean) adjusted for regres- 
sion within cells on the covariates PREI, PREC, SD2, and 
SD3 (cf. sections 15.3.5 and 15.4 above). The observed co- 
efficients, of partial regression within cells (b) and their 
corresponding t-values for testing the null hypothesis 
(b = 0) are as follows. 



PREI 


PREC 


SD2 


SD3 


0.47 


0.79 


2 


1.38 


3.7^ 


2.1^ 


6'.5^ 


3.2*^ 



All four t-values are significant, which means that all the 
covariates contribute significantly to the regression equa- 
tion. The coefficients for PREI, SD2, and SD3 are roughly 
equivalent to the corresponding coefficients obtained in the 
analysis of learning effect (El; section 16.1 above), which 
means that the individual pretest score -component (PREI) and 
the individual's intelligence (SD2, SD3) influence long-term 
learning about as much .as they influence immediate learning. 

The regression coefficient for PREC, however, is significant 
here, although-lt was not in the previous analysis. The 
class component in pre-treatment K!iowledge seems, then., to 
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Influence long-tenn learntng - retention - more than i.t 
influences iimiediate learntng. Perhaps immediate learning 
from the program 'suppresses' as itlwere the effect of pre- 
vious Instruction given iji the clas?, and previous learning 
Increases again in relative importance as the retention of 
learning from the program diminishes over time. It is also 
conceivable that learning relevant to the PUSL test, occur- 
ring within the classes after the P],*is more highly cc 
lated with the level of pre-treatment knowledge in the 
class, than it is with tie effect of the prograrmied in- 
struction, ' 



All cell means In the variable E2 i 
X S are given in table 16 



ri the design R x F x G 



Table 16.4 Long-term learning effect (E2); 

in the design R x F x ^ x S. 



cell means 



G 


S 


(n 


Rl 
^ F.5 


FO / 


/ 

' Fl 


R2 
F.5 


FO 


4 


M 
F 


7.86 
9.68 


11.25 
9.07 


5. si 
13iQ6 


10.88 
6.10' 


0.41 
4,73 


7.43 
6.69 


5 


M 
F 


6^60 


9.75 
12,68 


/57 
^i0.40 


12.97 
6,84 


18.84 
'8.30 


8.31 
4.57 


6 


M 
F 


12.09 
16.63 


18.98 
13.23 


11.18 
22.57 


11. &1 
15.02 


17.06 
14.51 


11.18 
14.21 


4-6 


M.F 


10.41 


12.58 


13.55 


10.57 


10.64 


9.52 



Ifi-Z.l Variance analysis of the design R x F x G x S 



The results of the variance analysis of the E2 cell means 
are summarized in table 16.5. The interactions 
R X F and R X G have been divided into linear and quadratic 
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components, which are also included in table 16.5. 



On the whole, the variation in experimental conditions has 
not entailed any great variation in long-term learning ef- 
fect (E2). In that respect the results of the analysis of 
long-term learning correspond well with those of the imme- 
diate learning effect (El; section 16.1 above). 

The only s-ignificant F-quotient for E2 involving R or F is 
the R X S 'interaction, which Is .illustrated in figure 
16.6 . The overt response requirement (Rl) - compared 
to the covert response requirement (R2) - seems to affect 
boys' long-term learning negatively or not at all, whereas 
girls' long-term learning is clearly afffected positively. 
To the extent that girls in grades 4 - 6 are more mentally 
mature than boys in the same grades, this finding is cdnt- 
rary to the hypothesis that the positive effect of overt 
responding decreases with increasing mental maturity. It is * 
possible, of course, that boys and girls differ in their 
'mode of learning' in a PI situation. If so, the nature of 
their different 'modes' remains to be identified. Whatever 
the explanation may be, the R x S interaction found here 
indicates that the effect of a response requirement may de- 
pend on other variables - in this case the sex of the stu- 
dent. , 

In view of the hypotheses formulated in chapter 7 the sour- 
ces of variation R, R x F, and R x G are particularly inte- 
resting. These variation sources are discussed below. 

Response requirement (R): The diffe- 
rence between response requirements as to long-term learning 
' (E2) points in the same direction as the corresponding dif- 
ference in immediate learning (El) : the overall mean is 
higher for RT than for R2 groups. The R1-R2 difference is, 
however,, smaller for E2 than for El, which means that it 
gives no support for a hypothesis that overt responding 
should have a more positive influence on retention than on 
immediate learning. Such a hypothesis would be reasonable, 
to the extent that overt responding Is believed to 'consoli- 
date ' learning. The result of yfhe E2 analysis is comp^atible, 
though, with the hypothesis formulated in this study, which 
expects the overt response requirement (Rl) to lead to a - 
compared to the covert response requirement (R2) - rather 



TabU 16,5 Long-^term learning effect (E2): variance ana- 
lysts of cell rfteans'in the design R x F x G x S, 



Source of 
variation 


df 


2 


F 




R ^ 

F 

G 

s 


1 
2 
2 
1 


33.66 
4.70 
171,34 


1.87 
<1 

9. 51*** 


3.88 
3.04 
3.04 

<J •oo 


R X F 
R X G 
R X S 
F X G 
F X S 
G X S 


2 
2 

1 

4 ' 

2 ' 
2 


13.13 

93.80 
8.97 
30.23 
, 3 .26 


■'<i 

5.21 
<1 

1.68. 
<1 


3.04 

3.88 
2.41 
3.04 
3.04 


R X F X G 
R X F X S 
R X G X S 
F X G X S 


4 
2 
2 

^ 4 


18.53 
22.67 
22.21 
8.35 


1.03 
.1.26 
1.23 
<1 


2.41 
3.04 
3.04 
2.41 


R X F X G X S 


4 


6.54 


<1 


2.41 


Error (within 
tells) 


272 


18.01 







Lwear and quadraHa aanponenta of-aertatn interaationa : 



Inter- 
action 


Component 


d 


^d 


^ ' *d 


R X F 


Linear 
Quadratic 


-'25.10 
-0.02 


20.77 
36.04 


-1.2 
-0.0 


R X G 


Linear 
Quadratic 


14.35 
10.87 


21.36 
34.99 


+0.7 
+0.3 



ir»o 



small increase in long-tenn learning effect. 



Interaction between response 
require ID, ent and key answer fre- 
quency ( R X f ) : The R X F interaction* is far from 
significant. The observed trend is almost totally linear . 
(table 16.5) and very similar to the one found in the ana- 
lysis of El, only less, marked. The interaction 1s illustra- 
ted in f i g u r e 1 6 . 7 . The hypothesis that decrea- 
/Slng key answer frequency (F) entails increased retention 
/when overt responses are required (Rl),' but decreased re- 
' tention when covert responses ^re required (R2)-, is not ve- 
rified. The observed cell means are, however, not at all 
Incompatible with this hypothesis. 

Interaction between response re- 
quirement and grade level (RxG): 
The RxG interaction is also far from significant, neither 
1s It clearly linear (table 16.5). It Is Illustrated in 
f 1 g U' r 16.8, which shows that the positive effect 
of the overt response requirement (Rl), if any, is still 
greatest at the lowest grade level (4). So the hypothesis 
that the positive effect on retention of Rl - as compared 
to R2'- Is greater the lowef^ the grade level gets little 
support from these data, even if It is not Incompatible with 
them. ^ - *:x 



16.2.2 Grade by grade analysis of the R x F x $ design 



As for El, a grade by grade variance analysis of E2 has 
been performed on the R x F x S design. Its results are 
summarized in table 16.6, which contains no sig- 
nificant F-quotlents for gra,de4 or grade 6. 
For grade 5 , only the R k S interaction is signifi- 
cant, with a trend quite similar to that found In the cor- 
responding analysis of learning effect (El; section 16.1.2 
and figure 16.5 above), as can be seen in figure 
16.9. Thus, the grade by grade analysis of long-term 
learning effect (E2) shows the same tendencies as the cor- 
responding analysis of immediate effect (El), with the only 
notable difference that the variation between subgroup means 
is, on the^ whole, smaller. 



4 




Figure 16,6 ^ , ' 

Long-term learning effect 
(E2): R X S 



£2, 




^ure 16.7 

Long-term learning effect 
(E2): RxF 




•Figure 16.8 

Long-term learning effect 
(E2): R X 6 . 



F/ KS- FO 

'Figure 16.9 ^ 

Long-term learning effect 
. (E2): RxF for grade 5 



16.3 PI TIME 



\ 

The variable LPT, i.e. the logar:.ithm of the PI time in mi- 
nutes, \is used as PI time measure (cf. section 15.3.3 
above). ^artiarregression coefficients (b) and the corres- 
ponding tvvalues (t^) for the covariates PREI, PREC, SD2, 
and SD3 (With LPT 'as variate have been computed with the fol 
lowing results: 

PREI PREC SD2 603 " 



b -.001 .002 -.001 -.001 

t. -1.1 0.7 -l.f -1.0 



All the above' t-values are .far from significant, and the 
analysis of LPT has, therefore, been performed without ad- 
justment for regression on the covariates. The cell means 
are given in t a b I.e 16.7, expressed in the origi- 
nal scale, i.e. transformed back to PI time in minutes (an- 
tilogLPT). A summary of the variance analysis of the R x F 
X G X S design is found in t a b ] e 1 6" . 8 , which 
shows three significant F-quotients: for R» for R x F, and 
for R X G. ^ ' ' 

In view of the hypotheses regarding PI time formulated for 
this study (table 7.1 above), the variation, sources R, R x 
F, and R x G - which also yielded significant F-quotients - 
are of particular' interest. 

Response requiirement (R): As expected 
the PI time varies with the response" requirement. The o\?erall 
mean time for Rl groups is 98.8 minutes, as compared to -49.8 
minutes for R2 gi-oups. This gives a PIT quotient of 1.98, 
i.e. the, overt response requirement (Rl) has led to twice as 
long PI time as the covert requirement (R2). The hypothesis - 
that PI time is greater under Rl than under R2 is, then, 
strongly supported, and so is the additional hypothesis that 
Pl'time under the Rl condition is 1.5 to 2 times the PI time 
under the R2 condition (cf. table 7.1 above). 



Table 16.? PI time (PIT): cell means in tfie "design 
R X F X G X S. 



G S 


FT 


Rl ' 
rv 1 


FO 


FK 


DO \ 

F.5 ' 


\F0 


4 M 

' F 


116.4^ 
97.2 


ni9.7 

124.2 


126.-6 
106.7 


52,6, 
52.5 


45.2 
48.6 


5^3 
55^ 


5 M 
F 


93.4 
8L4 


112.5 
111.4 


109.3 
107.3 


59.1 
47.5 


54.2 
55.4 


58. 2\ 
48.5 


6 M 
F 


72.0 
84.3, 


90.1 
79.2 


'93.3 
78.9 


47.0 
44,4 


''4'0'.8 
48.4 


44:0 
44.7- 


4-6 M,F 


89.8 


104.9 


102.6 


50.3 


48.5 


50.5 



Interaction between r«spo'nse 
requirement and key answer fre- 
•^.n^^ ( '^Z n • The R X F interaction is signifi- 

onl/H analysis of its components has yielded about 
aithL^h" ^^'^^^^ ^"^ ^^e quadratic components, 

although onlv the latter is significant (t - 2.1; table 
Interaction is graphically illustrated in two 

fof H]?f„! >V ''A ^ \'. ^ ° '■ ^" the form of PIT means 
Tor ditterent R and F combinations, and ir? the form of PIT 
quotients (R1/R2) for different values of F 

■ ?«c"w?y^''5 '-espbnses are required (Rl). the PI time increa- 

■ If decreasing key answer frequency from 100 (Fl) to - 
•su (F.5) per cent. When key answer frequency is further de- 
ulll P^*" PI timeldecreases somewhat. 
Ku? ^hl^^*/^f°"'?? t''^ required (R2). the PI time is a- 
bout the same for all key answer frequencies, only sliqhtlV 
shorter for F 5 than for Fl and FO (figure 16.10a). The noJ- 
linear character of the interaction is most clear y percei- 
ved in figure. 6.10b.- which shows how the PIT quotient for 
R1/R2 varies with F. The increase^in PI time entailed by the 
overt response requirement, expressed as percentage of PI 
.time under the covert requirement, is 79 per cent for Fl, 



Tcible 16.8 PI time (PIT): variance analysis of cell means 
in the variable LPT. 



Source of 
variation 


Hf 

U 1 


.2 


r 


F 


JR 


1 


.8001 




0 .OO 


F 


2 


*0032 


2.3^ 




f\ ■ 

u 


9 
c 




?5 38 


3.04 


S 


1 


.0032 


2.30 


3.88 


R X F 


2 


.0054 


^3.91'' 


3.04 


R X 6 


2 


.0060 


4.34 


3.04 


R X S 




.0011 


<1 


3.88 


F X G 


Y 


.0011 


<1 


2.41 


F X S 


2 


.0022 


1.5P 


3.04 


G X S 


2 


.0012 


<1 


3.04 


R X F X G 


4 


.0009 


<1 


2-. 41 


/ R X G X S 


2 


.0010 


<1 


3.04 


R X G X S 


2 


.0017 


1.21' 


3.04 


F X G X S 


4 


.0008 


<1 


2.41 


R X F X G X S 


4 


.0017 


1.19 


2.41 


Error (within 


276 


.0014 






cells) 







Linear and. quadratic aomponenta of certain interaationa : 



R X F. 
R X G 



apSSn, component,. d t^ 



Component. 


d 


^d 


Linear 


-.337 


. .182 


Quadratic 


-.663 


.316 


Linear 


.499 


.187 


Quadratic 


.345 


.307 



-1.9 
-2.1*' 

+2.7'** 
+1.1 



116 per cent for F.S, and 103 per cent for FO. * 

The hypothesis that, when overt responses are required, de- 
creasing key answer frequency leads to increasing PI time 
•1s» then» supported by these findings. The hypothesis seems 
tojieed some modification, though, since the PI time under 
theiRl concjition has its maximum when the key answer f re- - 
quency Is 50 per cent (F.5) and not when it is 0 per cent 
^(FO). Still, the PI time under the RIFO .condition is con- 
siderably, longer than it is under the RTFl condition. 

Interaction between res't)onse re* 
quirement and grade level (RxG): 
The interaction R x fi is significant and primarily linear 
(table >16,7). This can be seen in figure 16.11, 
where it is Illustrated in the form of PIT means for all 
R and G combinations, and in the form of PIT quotients (Rl/ 
R2)7or the three grade levels (G). The difference in PI 
time between Rl and R2 groups decreases with increasing 
grade level , which' means that the hypothesis formulated for 
this Interaction (table 7.1 above) is unequivocally suppor- 
ted. 

As figure 16.11a shows, there is a marked correlation bet- 
ween PI time and grade level under Rl condition, with time 
decreasing as grade Increases, Under the R2 condition, on 
the other hand^ PI time means for different "grades are much 
more similar. 



16.4 LEARNING EFFICIENCY 



The problem of measuring learning efficiency has b^en dis- 
cussed in different contexts (sections 2.5, 3.3.2.4, and 
9.7.3) and two alternative methods of estimating learning 
efficiency under different experimental conditions have been 
suggested: 

(1) The varial)le PI time (PITfi if necessary transformed in 
order to normalize its distribution, is used as covaria- 
te, which gives' as ^efficiency measure (EFF) the effect 
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Figure 16.10 PI time (PIT): R x F 
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Figure 16.11 PI time '(PIT): R X G 
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measure (E) adjusted for regr^sston on PIT, i.e. 

EFF « E - bpj^(PIT..- PTT) , where bpj^ 1s the 

coefficient of partial regression within cells for PIT 
(cf. 9.7.3 above). . 

..(2) A conslusion chart 1s used for determining the relative 
efficiency of different experlmehtal conditions as to 
learning effect and, PI time (section 3.3.2.4 above). 



16.4.1 Variance analysis of EFFl ; the design 
"fe X F X G X S ^ 

' My.* 

Of the tWQ suggested methods for esftlmating efficiency the 
first one seems preferable when the necessary conditions 
fmpUed In^lt are satisfied; particularly the condition 
that the partia-l regression coefficient for PIT Is greater 
than zero (section '9.7.3 above). This method of estimating 
efficiency only accounts for the linear regression of lear- 
ning effect on PI time, although 1t cannot be taken for 
granted that this relatiolhship Is linear (section 3.3:2.4 
above). Since PIT is transformed here to LPT, which is ap- 
proximately normajly distributed (figure 15.2 above), the 
assumption of Jtnear regression seems, however, tenable. 
* ^' 

The efficiency measure discussed here (EFFl) has been ob- 
tained by me^s of adjusting the variable EDI for regres« 
sibn on ther previous covariates PREI, 'PREC, SD2, and SD3, ' 
plus the covariate LPT, i,e. 

EFFl ^ EDI - 2:b^(x - X) , where x = PREI, PREC, 

SD2, SD3, and LPT, and b is the partial regress16n coeffi- 
cient within cells for x? 

The following values have been computed for' these partial 
\ regression coefficients and their corresponding^ t-values 
> for testing the null hypothesis (b = 0). 



ERIC'' 



\ 

m * 



PREI PREC SD2 SD3 LPT 

b o!50 ,0.18 3.34 1.31 16.64 
+4.2^ +0.5 +8.2**" +3.3^'' +2.6*^ 

b : 



The coefficients for the four covariates involved in the 
computation of the effect measure (PREI, PREC, SD2, and 
SD3) are very similar to those obtained in the analysis of 
learning effect (section 16.1 above). The regression coef- 
ficient for LPT is significantly positive, which means that 
the variable EFFl i^s cyi ^ccJeptable estimate of learning ef- 
ficiency in the present study. EFFl has been analysed in 
the same manner as th? measures of learning effect (El) and 
retention (E2). The cell means in EFFl in the design R x F 
X G X S are found in t a b 1 e 1 6 . 9 , and the varian- 
ce analysis of this design is summarized in t a b 1 e 
16.10. 

The variations in experimental conditions do not seem to . 
have entailed any great differences in learning efficiency. 
Except for the interaction R x G x S, which is extremely 
difficult to interpret,: there are no significant F-quotlents 
involving the factors R or Fv(table.l6.]0). In view of the 
hypotheses that have been formulated conterning learning ef- 
ficiency (table 7.1 above), a closer inspection of the varia- 
tion sources R, R x F, and R x G is,vhoweYer, needed. 

Response requirement (R): The F-quo- 
tient for R is not significant, but it is fairly close to 
the limit value (3.39 as against 3.88). The overall EFFl 
mean for Rl groups is 12.24, and for R2 groups 14.67. The 
observed efficiency is, then, higher when covert response 
are required (R2) than when overt responses arfe required 
(Rl). This is* in agreement with the hypothesis formulated a- 
bove (table 7.1): covert response requirement leads to more 
efficient learning than does overt response require^nent. 

Interaction between response r.e - 
quifement and key answer frequen 
c y ( R x F ) : The^ R x F interaction is predominantly li- 
near, but it is not significant (table 16.10). The interac- 
tion is illustrated in figure 1, 6 . 1 2 , which 
shows that the efficiency is higher in R2 groups than in -Rl 



h/''''"'"'^.!^"^^ condition (table 7.1 abo 

ve), and decrease with the key answer frequency under the 

S2 dSJL^Jl! ^Pothesis. except- perhaps in one respect: 
;?Mo„r response requirements (R) as to ef- 

fic ency is smallest for the lowest key answer frequency 

ffithesMTfo'f^o'S?''^^' *° 

The increase in efficiency entailed by R2 as compared 

td^ '"^'"'^^^ '^^^ tfi^ '<ey answer frequency is 

high, and least marked when it is zero. 

I n t e r a c t,1 0- n . be t w e e n, response re 
q u i r e m e n t ,a n d grade 1 e v e 1 ( I y r \ . 
This Interaction is almost completely linear, but far from" 
significant (table 16.10). It is illJstfated in f 1 g IT 
h fLl Ll ot^""^ ^^""^ EFFl value is highe? for 

R2 than for Rl groups at all grade levels. The difference 
between RT and R2 groups in tenns of ^FFl i . howevl?! smal 
lest in grade 4 and greatest in<grade 6. so the resSlt U 

TotU .... tearning^efficiency^(EP>l^^ cell means in 



Fl 



Rl 






F.5 


Vo 


- Fl 


9.97 


9.33. 


11.11 


6.99 


11.66 


11.79 


7.85 


n.39 


17.13 


12.15 


20.54 


11.74 


13.50 


16.50 


18.21 


12.73 


21.94 


19.22 



R2 

F.5 FO 



c ^'33- 11-11 5.86 10 15 

F 10.46 6.99 11.66 11.79 8.45 lliso 

M 4.69 7.85 11.39 17.13 21 13 f? 57 

F 13.02 12.lt; ?n W n 7^ .nit 



c "'•"0 18.21 16.97 20 44 

F 16.49 12.73 -/I QA iQ oo o" ,0 



12.67 t3.04 

16.97 20.44 
24.12 23.03 



M.F 10.97 10.53- 15.23 14.87 14,88 14.26 



TdbU 16.10 Learntng efficiency (EFFI): variance ana- 
lysis of cell means. 



Source of 
variation 

R 
F 
G 
S 



R 
R 
R 
F 
F 
G 

R X 
R X 
R X 
F X 



F 
G 
S 
G 
S 

S ' 

F X G 
F X S 
G X S 
G X S 



df 

1 
2 
2 
1 

2 
2 
1 
4 
2 
2 

4 
2 
2 

4 



R X F X 6 X S * 4 

Error (within p?-! 
cells) • 



52.95 
15.04 
234.41 
22.53 

26.12 
9.92 

'33.56 
6.48 
4.28 
3.95 

17.10 
. 9.59 
51.35 
2.57 

/ 2.15 
15.62 



3.39 

IS^Of" 
1.44 

1.67 
<1 

2.15 
<1 
<1 
<1 

1.09 

3.29*^ 
• <1 



3.88 
3.04 
3.04 
3.88 

3.04 
3. 04 
3.88 
2.41 
3.04 
3.04 

2.41 
3.04 
3.04 
2,41 

2.41 



Lineax' and quadratio oomponenta of aertccin interaaHona 



Inter- 
action 


Component 


d 


^d 


^d 


R X F 


Linear 
Quadratic 


-29.24 
34.58 


19,3J 
33.56 


-1.5 
+1.0 


R X G . 


Linear 
Quadratic 


21.76 
-2.74 


19.89 
32.59 


+1.1 
-0.1 

7f 



.The interpretation of the R -x G interaction is complicated 
%^^n case with the learning effect analysis (sec- 

tion 16.1.1 above) - by the significant interactld'n R x G 
X S, Vhich is illustrated in f i g u-r e 16 1 4 As 
this figure shows, R2 has been more efficient than RlVor 
both sexes in grades 4 and 6, whereas in grade 5 R2 Has 
been most efficient with boys and Rl with girls. It has al-" 
ready been pointed out that suth 'an^interaction could not • 

I'O'-eseen on the basi.s of previous research, and 
ri§ satisfactory explanation of it can be presented here. 

HhS^w^JS^^s;.*!!^ findings presented here -are not incompa- 
tible with, the hypothesis that the superiority of dovert 
response requirement - as compared"^ to overt - in terms of 

S^nflfc'^n^'i'"^'!!'' ^^9*^^^ *he grade level of the 
students. On the pther hand, the findings do not give any* 
strong support t.oy;he hypothesis. " ^ 

16.4.2 . "Grade by gra de analysis of the R x F x S design 

The efficiency variable". EFFl . has also been subjected to 
a grade by grade analysis, performed on the design R x F 
X S. The resu ts of this analysis are summarized in tab- 

iL-^Jn^ tt quotients are less than 1. i.e. the 

variation between grpUps fs small compared to that within 
groups. It should be, remembered that-the analysis of lear- 

D^^-iE-^ ^^^^'^^'^ ^ significant F quotient for the j 
in Ri fVn^il ll 9'"ade. meaning .that the effect was higher 
in R.l than,in R2 groups (section 16.1.2 above). There is. 
however, no corresponding. difference in terms of efficien- 
cy. Qn the contrary, the EFFl mean for the overt response 
requirement groups (Rl) is slightly lower than that for the 
covert response requirement 'groups (R2). viz. 9.09 as com- 
Sff!^^. • ^-^^-A '^?"?eQuently. .the- fa.ct that the learning 
tZ^ity ^''^'^t 4 higher under the Rl than under the R2 
?no f^^^ explained by the difference, in work-" 

ing time (PI time) entailej by the two conditions. 

In g r a d e .5 the R^x F and R x S interactions are sig- 
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Figure 16:^4 Learning efficiency (EFFl): R x G ^ S 
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Table 16.11 Learning efftctency (EFFl); grade by grade 
variance analysts of the design R x F x S, 



Gra- Source ^2 r- c 

de of var. at s ^ ^ct 



4 


R 


1 


1.42 


<1 


3.88 




r 


z 


O 0*7 


<1 


3 .04 




c 
o 


T 
1 


c yiQ 
b.4o 


<1 


3.88 




R X F 


2 


5,19 


<1 


3.04 




D V C 

K X o 


1 




<1 


3 .88 




C v C 

r X o 




1 .94 * 


<1 


3.04 




R X F X S 


2 


5,70 


<1 


3.04 


5 


R 


1 


15.50 


<1 


3.88 




r 


o 

<: 




< 1 


3.04 




S 


1 


0.96 


<1 


3.88 




R 'x F 


2 


47.64 


3.05* 


3.04 




R X S 


1 


134.40 


8.60** 


3.88 




f X S 


2 


5.43 


<1 


3.04 




R X F X S 


2 


n.n7 


<1 


3.04 


6 


R 


1 


55.86 


3.58 


3.88 




F 


2 


15.90 


1.02 


3.04 




S 


1 


24.00 


1 .54 


3.88 




R X F 


2 


7.85 


<1 


3.04 




R X S 


1 


1.82^ 


<1 


3.88 




F X S 


2 


2.40 


<1 


3.04 




R x*F X S 


2 


7.75 


<1 ■ 


3.04 


4-6 


Within 
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15.62 
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Figure 16. IS Learning efflcienfcy (EFFl): ' R x F 
for grade 5 



nificant, i.e. the same interactions that yielded signifi- 
cant F cluotients in the El analysis (section 16. T. 2 above). 
The interaction R x F, which is illustrated in f 1 g u 
re 16.15, implies that "IQO and 50 per cent key an- 
swers (Fl jand F.5) lead to higher efficiency under the R2 
condition than under the Rl condition, whereas 0 per cent 
key answers (FO) has the Opposite effect: higher efficien- 
cy under Rl than under RZ. It is worth noting in this con- 
text that the traditional cotnbination of maximum amount of 
"active responding" and information about correct answers 
(RlFl) as well as the combination of minimum amounts (R2F0) 
both Seem to give lov/er efficiency of learning than most of 
the other combinations of R and F. , 

In g r a d e 6 there is no significant F quotient, al-- 
though the value for the factor R is close to the limit 
value, viz. 3.58 as against 3.88. The analysis of learning 
effect (El) showed practically no difference between the Rl 
and the R2 aroiips in this grade: means were . 17. 23 and 17.09 
respectively. The observed difference in terms of efficien- 
cy (EFFl) is, however, much greater with lower efficiency 
in the Rl groups than in the R2 groups; EFFl means 16,03 
and 20.34 respectively. This finding is in agreement with 



the hypothesis of htgher effictency under covert (R2) than 
under overt (Rl) response requirement, especially in the 
higher grades (cf. chapter 7.1 above). 



16.4.3 Efficiency estimates based on special conclusion 
chart ~ 



The relative efficency of learning (EFF) under different 
response requirement conditions can also be evaluated by 
means of .the conclusion chart suggested in the research re- 
view above (section 3.3..2.4 above). Since there is a ten- 
dency of interaction fcet?/een response requirement (R) and 
grade level (G) in terms of learning effect (El; section 
>16,1.1 above), this evaluation had better be made for each 
girade level separately. 

«» ■ ' 

The following relationships have been found betv/een learning 
effect (El) and PI time (PT) mean for Rl and R2 groups in 
the different grades. 



Grade 


4 : 


EURI) 


> E1(R2) 






PT(Rl) 


> PT(R2) 


Grade 


5 : 


EURn 


= E1(R2) 






PT(Rl) 


> PT(R2) 


G. r a d e 


6 : 


E1(R1) 


= E1(R2) 






PT(Rl) 


> PT(R2) 



Note that = means 'not significantly different from', and > 
'significantly greater than". 

From these premises the following conclusions can be drav/n 
concerning the relative efficiency (EFF) under the Rl and 
R2 conditions in each grade. 

Grade 4 : EFF(Rl) ? EFF(R2) 

Grade 5 : EFF(Rl) < EFF(R2) 

Grade 6 : EFF(Rl) < EFF(R2) 

* Thus, efficiency of learning is apparently lower under tlie 
overt (Rl) than under the, covert (R2) response requirement 
i^ grades 5 and 6, wherea? in grade 4 the difference betv/een 



these response requirements in terms of efficiency is unde- 
termined. These findings lend a great deal of support. to the 
hypothesis that the Rl condition leads to lower efficiency 
of Vearaing than the R2 condition. The findings are also 
compatible with the hypothesis that this difference between 
response requirements in terms of efficiency is greater the 
grade level of the students. 



16,5 THE 'STANDARD' RlFl CONDITION COMPARED WITH^ 
OTHER EXPERIMENTAL CONDITIONS 



The standard combination, in Skinnerian PI, of overt re- 
sponse requirement and l(fE)*per cent key answer frequency, 
i.e. the equivalent of condition RlFl in the present Study, 
is often asstimed to provide good, if not optimal conditions ^ 
for learning in most PI situations. It is not always made 
clear, though, if the criteria of good learning conditions 
refer only to the effectiv^-ness of learning, or if they al- 
so take efficiency into account. In the present study both 
effectiveness and efficiency are -considered important in 
attempts to evaluate the outcomes of PI. 

As a kind of 'control' hypothesis (cf, chapter 7 above) it 
has been assumed in this study that the RlFl combination 
of response requirement and key answer frequency "is supe- 
rior to all other RF combinations investigated here, both 
in terms of immediate (El) and long-term (E2) learning ef- 
fect and in terms of learning efficiency (EFFl), This hypo- 
thesis has been tested by means of variance analyses of the 
three variates (El, E2, and EFFl) on the design (RlFl - all 
other RF)' x G x S, whicK has 2 x 3 x 2 = 12 cells. 

The results of these variance analyses ai^e summarized in 
tables 16.12, 16.13 a n d 16,14, As 
can be seen in tables 16,12 and 16,13, there is no signifi- 
cant varjati-on associated with the factor RF, neither in El 
nor in tZ, However, the overall mean for RlFl groups is in 
both cases - El and E2 -lower than the total mean for all 
other RF groups: 12,82 against 13*60 in El , and 10.41 a- 
gainst 11,37 in E2, Clearly, then, the standard Skinnerian 
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Table 16.12 The combination RlFl compared with all 
other RF combinations in terms of lear 
rring effect (El), 



(a) Variance analyaia reaulta 



Source of ^2 
variation ' ^ 



RF 



RF X G 
RF X S 
G X S 

RF X G X S 

Error (with 
cells) 





1 


1.83 


<1 


3.88 


2 


• 43.46 


•4.7^** 


3.04 


1 


18.63 ^ 


2.03 


3.88 


2 


4.82 ' 


• <1 


3.04 


1 


10.14 


1.10 


3.88 


2 


0.32 


' <1 / 


3.04 


2 


5.63 


<1 


3.04 


272 


9.18 







Note: RF = R1F1 - all otheX Rf 



(b) 


Cell 


means 




G 


S 


RlFl 


Other 
RF's 


4 


M 
F 


9.72 
12.78 


9.53 
10.15 


5 


M 
f 


7.04 
14.37 


14.48 
12.89 


6 


M 
F 


15,21 
17.81 


15.82 
18.75 



4-6 M,F 12.82 13.60 
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Table 16.13 The combination RIFT compared with all 
other RF combinations in terms of lear- 
ning retention (E2). [ 

^(a) Varianae analye'ie reaulte 



Source of 
^variation 


df 


7ZP^ 

s2 F 


/a 


RF 

G 

S 


A 
2 
1 


2.77 
50.16 
8.55 


4.84** 

<1 


3.88 
3.04 
3.>88 


RF X G 
RF X S 
G X S 


f. 2 
1 
2 


4.47 
6.18 
0.89 


<1 
<1 
<1 


3.04 
3.88 
3.04 


RF X G X 5 


2 


5.03 


<1 


Q HA 


Error (within 
cells) '^''^ 


10.37 






Note;. RF = 


R1F1 - an 


other RF 






(h) Cell mans 








G S 


R1F1 


Other 
RF's 






4 M 
F 


7.86 
9.68 


7.10 
8.03 






5 M 
F 

6 

F 


6.60 
9.61 


11.69 
10.56 






12.09 
16.63 


14.95 
15.91 






4-6 M,F 


10.41 


11.37 
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!rcd)le 16,14 The combination RIFT compared with all 
other RF combinations in terms of lear- 
ning efficiency (EFFl). j 



fa) Variance analyeia reaulta 



Source of 
variation 


° df 


.s2 


F 




RF 


1 


37.28 


4.14^* 


3.88 


G 


2 


75.79 


8.42*^ 


3.04 


S 


1 


18.43 


2.05 


3.88 


RF X G 


2 


6.31 


<1 


3.04 


RF X S 


1 


5.37 


<1 


3.88 


G X S 


- 2 


0.03 


<1 ' 


^.04 


RF X G X S 


2 


3.85 


<1 


3.04 


Error (within 
cells) 


271 


9.00 







Note: RF = RlFl - all other RF 



(h) Cell means 



G 


S 


RlFl 


Other 
RF's 


4 


M 


6.12 


9.28 




F 


10.46 


10.04 


5 


M 


4.69 


14.01 




F 


10.32 


13.57 


6 


M 


15.01 


17.12 




F 


16.49 


20.22 



4-6 M.F 10.52 14.04 



\ 



comfainatton RlFl -fc -fn 

nan condition of ow^Lr ^^''"'^^ ""^er the standar^'c?- 
This analysis gives fh^n 

hypothesis that the 'Skinn'"° '"PP°'"* whatsoever fo. fh 
tetter iSJn^niJfcJInSf ^'^""-^^ ^^ttlo^^^^^ - , 

PROGRAM RESPONSE VARIABLES 

pX?a^ Ss'l.? "th^^ri S^r""^*^^ *° ^^^Pond overt! V to 
computed in the fnnl° ^" groups, Valued ha, yf ^ '^ *° 
bove). '^'^ ^°"°^n9 variables (cf. section iLa^fa- 

e°r;jff."^^^- °^ o^^^inal pro. 
2d ^y°^t\?S„3:,?,^3er Of errors correc 

SrrS^^SoJsV^^ °^ ---^ng. un- 

the inverse sinp of fk« 
corrected by HI s^'uie'n\'h7Klj?" ^^--^ 



iClV 
APC 



tei 



.atleie.lS Analysis of the variable LVl. 
(a) VaHanoe amlyBia veaults- 



1.72 





5ir 
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Table 16^X6 'Analysis of the variable LCW. 
(a) Varianae analys-Ce reaulta 



Source of 
variation 


s2 


F 


'a. 


F 2 
G 2 


0.1552 
0.0025 


26.21^ 

<1 . 


3.06 
3.06 


F X <G 4 


0.0038 


<1 


2.43 


Error (within 
cells) '^^ 


0.0059 






(b) Cell tneane of LCV 








G FT F.5 


FO 






4-6 0.943 0.887 


D.524 







16,6.1 Variance" analyses of the design F x Q 



Variance analyses with the design F x. G have been perfor- 
med for all four program response variables, LW,LCW, LUW, 
and APC, The results of these analyses are suimnarized in 
t a b 1 e s 1 6 , 1 5 - 18, As these tables show, 
all the program responses variables analysed here covary 
with the key answer frequency. Number of errors (LW and 
LUW) increases with decreasing key answer frequency,. from 
Fl to F0,< while corrections (U.CW and h^Z) decrease. Tfie 
relationships between key answer frequencyi on the one 
hand, and the four program rasponse^variables ^ an the oth- 
er, are illustrated in figures 16,16 a^n d 
16.17, which show approximately linear relationships 
for the variables LW, I^UW and APC. 



TabU 16.17 Analysis of the variable LUW. 



(a) Variance analyeia reautta 



Source of 
variation 


df 


s2 


F 




F 
G 


2 
2 


0.5728 
0.1174 


75,37^ 
15,45"^ 


3.06 
3.06 


F X G 


4 


0.0086 


1.13 


2.43 


Error (within 
cells) 


144 


0.0076 




V 



(b) Cell means tranafomsd back into unoorrected 
errors in per cent of total number of r&sponeee 



G 


r 1 


F.5 


FO 




4 


3.0 


10.7 


24.0 


9.3 


5 


1.2 


6.3 


16.0 


5.2 


6 


1.4 


3.1 


9.4 


3.5 


4^-6 


1.7 


6.0 


15.3 





Three of the variables^ Uth tU|f and APC, also covary signi- 
ficantly with the grade level (G), as\ls illustrated in 
figures 16,18 and 16.19. Not unexpec- 
tedly we find that the nBmber of errors (LW and LUW) Is 
smaller the higher the grade level, while the relative fre- 
quency of corrections increases with increasing grade level. 

, These^analyses indicate that the variation in key answer 
^ frequency Influences the response behaviour of the students. 
The finding that tfie number of response errors increases 
when the key answer frequency decreases is fully in agree- 
ment with traditional, Skinnerian PI theory. As a matter otj 
fact, th§ frequency of correct program responses is some- 



Table 16.18 Analysis of the variable APC. 

\ • . ^ 

la) Variance analysta reeulta 



Source of 
variation 


df 


s2 F , 


'a 


F 

G . 


2 
2 


0.3327 79.59^ 
0.0255 6.09^ 


3.06 
3.06 


F X G 


4 


0.0017 <1 


2.43 


Error {within 
cells) 


144 


0.0042 




(b) Cell means tranafomed bask into correations 
per aent of the number of original errors 


G Fl 


F.5 


FO Fl-0 • 




4 51.3 

5 73.3 

6 69.6 


26.8 
35.8 
42.3 


3.3 23.7 
7.6 36.6 
11.9 39.8 




4-6 65.0 


34.8 


7.2 





times regarded as a measure of program quality, 4n the sen*- 
se that the learning effect should be greater the.hlgher the 
proportion of correct student responses in the program. 
This interpretation of a high frequency of correct student 
responses in a program is, however, not supported fay the re- 
sults of the present study. Although decreasing key ansv^er 
frequency has led to a decrease in the proportion of cor- 
rect student responses, this does not seem to have entailed 
any marked decrease in learning effect (cf. section 16.1 
'above) - * 

The students* spontaneous corrections .of their response er- 
rors (the variable APC) could also be expected to Influence 
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Figu^ 16.16 Hgure 16.17 

The variables lU, LUU and The variable APC: covaria- 

LCH: covariation vdth key tion with key ansv^r fre-- 
answer frequency (F) quency (F) 



u 
li 
n 

0.1? 




Figure 16.18 



The variables LW and LUW: 
covariation v/ith grade 1e- 
ve-nfG) 



APC 

o.t- 



<5V 



as 



as 



Figure 16.19 

The variable APC: covaria- 
tion with grade level (G) 
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learning. Other PI conditions being equal, e( higher amount 
of corrections should lead to a higher Teaming Ipffect, 



since it diminishes the possibility that incorreist respon- 
ses are learned and retained as correct ones. If^xhis is so, 
the learning effeQt^^in this study should be expected to de- 
crease with decreasing key answer frequency, since the pro- 
portion of corrections decreases with this variable (figure 
16.17). As we have*already seen (section 16.1 above) this 
is, however, not the case. It seems reasonable, though, to 
regard the fact that corrections decrease with decreasing 
key answer frequency as evidence that the students have u- 
sed - at least to some extent - the response information 
provided tn the form of key answers. 



16.6.2 Correlation analyses 



Differences betv/een groups in the different program re'spon- 
se vartables do not, as we have seen, seem to influence 
learning in the ways that are usually predicted. In order 
to throw more light upon this question the relationships <>• 
between the criteria EDI and*€D2, on the. one hand, and the 
response vartables IM, LC^, LUW and APC, on the other, have 
been analysed. For each of the nine cells In the design F x 
G the full matrtx of correlation coefficients for these six 
variables has been cotnputed. These coefficients are rende- 
red In t a 5 1 e 1 6 , 1 Sf , 

The correlation coefficients In table 16.19 have been trans- 
formed to Zr (Fischer's z] and for each pair of variables 
the observed variance between groups in z has been compu- 
ted according to the formula 

k k 

s^ (Zz^ - (Sz )^/k)/(k- 1), where k = number of 

2« 1 r 1 r nrniinc 



r 



groups. 
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The expected variance for the nine groups, assuming a ran-^ 
dom normal distribution of z , is 



where = number of subjects in group i. 

The observed variation between groups for z has been t^s 
ted using the test variable 



The observed values of this variance quotient are found in 
the bottom row of table 16.19. The variance quotient is 
non-significant for s1^ of the variable pairs; EDlxED2» 
EDlxLCW, ED2XLUV1, EDZxLCW, ED25^P£ and LUWxLCW. For these 
variable pairs the variation between groups in the observed _ 
correction coefficients is, then, not greater than could 
be expected if coefficients v^ere randomly sampled from a 
population with a mean equalling the observed mean. These 
observed mean correlations are also given in table 16.19, 
where one can see that three of these values (mean r) are 
not significantly different from zero, namely EDlxLCW, ED2x 
LCW and LUWxLCW, 

Apparently, the varia^ble LCV/ (errors corrected by the stu- 
dent) is uncorrected with £01 and ED2, i.e, the number of 
program response errors corrected by the student does not 
seem to influence the learning effect. The variable LCW may ^ 
indicate that the available response infonnation (key an- 
swers) has been utilized by the students, but it does not 
contribute to the explanation of variations- in learning ef- 
fect. The variable LCW is, therefore, excluded from the 
subsequent analyses. ^ 

The variables LW and LUW (original and remaining, uncorrec- 
ted program response errors respectively) correlate highly 
with each other in F.5 and FO groups. Coefficients between 
0^84 and 1,00 indicatie that the tv/o var-iables are practical- 
ly identical in these experimental groups. Within Fl groups 
the correlation between these two variables is only modera- 
tely high, and the correlations with the effect variables 
EDI and ED2 are higher for LUW than for LW. In analyses of 
relationships between program response variables and post- 



r 




k 

= i:x(l/(n 
• 1 



. - 3)/k = =-.0729, 
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Table 16. 2Q Partial correlations v/ithin cel]s in the design 
F X G faett/een APC and EDI vdtfl LUW held cons- 
tant. 



Group APC-EDl APC»ED2 Note: 

2 

Concerning and s^ see note 
in table 16,19. ^ 

ol = i:(l/(n. - 4))/k. 
r i=l 

where k = number of cells and 
n| = number of Ss in cell i. 



64 F1 


.161 


.109 


F.5 


.047 


.129 


FO 


-.579 


.575 


G5 Fl 


.149 


-.401 


F.5 


.046 


-.127 


FO 


.243 


.201 


G6 Fl 


.037 


.074 


F.5 


-.257 


-.131 - 


FO 


.082 


.161 




0.99 


i.n 








r 


-.017 


.072 
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test results the variable LUW seons, then* to be a more ade- 
quate measure than the variable LW, i.e. the remaining, un- 
corrected errors made by the student provide a better mea- 
sure than the original errors. 

The two program response variables having the highest cor- 
relations with posttest results (EDI and ED2} are conse- 
quently LUW» number of uncorrected errors, and APC, propor- 
tion of errors corrected by the student. These two variab- 
les are also fairly highly correlated with each other: co- 
efficients for the nine groups vary between -0.31 and -0.91 . 
(tab^e 16.19). Since LUW on the whole shows a higher corre- 
lation with EDI and ED2 than does APC* the partial correla- 
tion coefficients for APC and EDI - or ED2 - v/ith LUW kept 
constant have been computed. These coefficients are given 
intablel6.20. 

The variance quotient for Fischer's z, i>e. observed varian- 
ce divided by expected variance, is 0.99 for the partial 



correlation APCxEOl and 1.11 for APCxED2 (table 16.20). 
Neithe^r of these values is significant, which means that 
the vdHation between experimental groups in partial cor- 
relation for these variables is not greater than could be 
expected if coefficients were randomly sampled from a popu- 
lation with a mean equalling the observed mean. The mean 
r values computed, -0.02 for APCxEDl and 0.07 for APCxED2, 
are both nonsignificantly different from zero. The conclu- 
sion is, then, that there is no partial correlation between 
APC, on the one hand, and EDI or ED2, on the othert when 
LUW is kept constant. Of the four program response variab- 
les studied here only one, the number of uncorrected errors 
(LUW), seems consequently to have any particular value for 
the prediction of posttest results. 

The corrtelation , between LUW and EDI and ED2 respectively 
variesfias table 16.19 shoves, from one experimental group 
to another. The variation between groups in Fischer's z 
for these two variable pairs has been analysed by means of 
variance analyses on the design F x G, The results are sum- 
marized in tables 16.21 and 16.22. As 
these tables -show, the variance quotient (observed through 
expected variance) is significant f^r the factor F in both 
cases, but not for G or for the interaction F x G. The cor- 
relation between the number of uncorrected errors made by 
the students (LUW) and posttest results (EDI or ED2) varies 
then, with the key answer frequency (F), in that this corre 
lation 1s higher the, lower the key answer frequency (cf. 
r mean values in tables 16.21c and 16.22c). 

In order to clarify further what factors influence the va- 
riable LUW under different conditions as to key answer fre- 
quency, the standardized coefficients of the partial regres 
sion of LUW upon the covariates in the previous analyses, 
namely PREI, PREC, SD2, SD3 and LPT, have been computed fpr 
Fl, F,5 and FO groups respectively. These partial regres- 
sion coefficients (R) are found in table 16.23. 
The coefficients in this table must* of course, be interpre 
ted with' particular caution in view of the low number of 
degrees of freedom involved. They give, however, some inte- 
resting indications about the nature of the influence pre- 
treatment knowledge (PREI, PREC), intelligence (SD2, SD3) 
2bnd PI time (LPT) have upon the number of program response 
errors made by students. 



Table 16*21 Variatton between groups in correlation betv^een 
uncorrected errors (LUM) and posttest result 
(EDI). 

(a) z^"^alue& for Rl.gvoupa in the^jeaign F x G 



G 


Fl 


F.5 


FO 


Fl-0 


4 


■ -,487 


-1.088 


-1.221 


-.932 


5 


-.839 


-,.966 


^-1.705 


-1.170 


6 


-.059 


-.935 


-1.102 


-.699 


4-6 


-.462 


-.996 


-1.343 





(h) Vari'anae analyaia 



Source 
of var. 


df 


; s^ 


sV 


F 


2 


. 0.5910 


8.11^ 


K 


2 


0.1666 


2.29 


F X G 


4 


0.0473 


0.65 



(a) v-valuea 



Fl F.5 FO 

r -.430 -.759 -.880 



Table 16, Variation betv/een groups in correlation between-* 
uncorrected errors (LUH) and posttest 2 results 
(ED2). ^ 

(a) z -values for Rl groups in the design F x G 



G 


FT 


F.5 


FO 


FT-0 


4 


-.224 


-1.066 


-.994 


-.761 


5 


-.682 


-1.015 


-1.119 


-.939 


6 


-.116 


-.957 


-.959 


-.577 


4-6 


-.341 


-1.013 


-1.024 





(b) Variance analysis 



Source 
of var. 




df 


s^ 




F 
G 




2 
2 


0.4593 
0.0534 


6.30^ 
0.73 


F X G 




4 


0.0235 


0.32 


(q) r-'Values 




Fl 




F.5 


FO 


r 


.328 




-.767, - 


.772 
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TabU 16.23 The regression of im on PRE! , PREC, SD2, SD3 
I and LPT for different key answer frequencies 

(F; for Rl groups only). 



F 


°PREI 


^PREC 


°SD2 


°SD3 


^PT 


r2 


Fl 


-.080 


.122 


-.§86 


-.089 


-.507 


.309 


. F.5 


-.146 


.083 


-.684 


-.260 


.004 


.760 


FO 


.035 


.166 


-.575. 


-.390 


-.160 


.617 



Note: fa^ is the standardized (scale free) part-ial reg- 
ression coefficient for variable i v/ith the oth- ' 
er four variables held constant. 

R^ = hb'^.r . , where i = PREI, PREC, $D2,SD3, LPT 
i=l ^ y = LUW 

k = number of covariates 



When the key answer frequency is 100 per cent (Fl), LUW ob- 
viously depends largely upon other variables 'than those in- 
vestigated here, since these only account for 31 per cent 
of the LUW variance. The highest partial correlation - for 
Fl groups - is that between LUW and LPT {ydth a coefficient' 
of -0.51), which implies that the response errors in the 
program depend to a marked extent on the PI time. The corre- 
lation is negative, so the shorter the time devoted to the 
program the more numerous the errors made. It seems there- 
fore reasonable to assume that the errors made in the pro- 
gram and not corrected by the students, when the key answer 
frequency is 100 per cent, are primarily mistakes caused by 
neglect rather than errors caused by lack of knowledge. If 
this is so, one should not expect these errors - or mis- 
takes - to^ correlate highly with the posttest results (cf . 
tables 16.21-22). ^ 

When the key answemfrequency is 50 or 0 per cent (F.5 or 
FO), the amount of Uncorrected errors in the program (LUW) 
depends to a much gft^ater extent on the five covariates stu- 
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dVed here. The percentage of LUW variance explained by the 
covariates is 76 for F.5 and 62 for,F0, which should be 
compared to the figure 31 per cent for Fl. All covariates 
do- not, hov/ever, influence the number of errors left uncor- 
rected 6y the students to the .same extent. The PI time 
(LPT) seems" to be of very Tittle importmice in this context 
(the coefficient is 0-04 for F.5 and -0.16 for FO). But the 
uncorrected errors (LUW) are fairly highly correlated with 
Intelligence (SD2, SD3), particularly with verbal intelli- 
gence (SD2, v/ith the coefficient equalling -0.68 for F.5 
and -0.58 for FO). [ 

Sumn^arizing the findings presented here concerning the va- 
rious program response variables we note that 

- the covariation between the amount of corrections of 
program response errors and the key answer frequency 
provides evidence that the key answers have been used 
by the students for the "purpose of response control; 

-* the correlation between program response errors and post- 
test result is highly dependent upon the key answer fre- 
quency, in that this correlation is high only when the 
key answers have been partly or v/holly removed from the 
program; ^ 

- response errors made in the program are of a different 
type depending on the key answer frequency, particular- 

' ly in the sense that errors made under the condition of 
TOO per cent key answers are likely to fee mistakes (cau- 
sed by neglect), which is not the case under the condi- 
tions* of 50 or 0 per cent key answers; and 

- the responses a student is expected *;o give in the pro- 
gram 'The main parts of the simple sentence' mu^t on thfe 

* whole be considered criterion relevant, sincfe the corrre- 
lation between response errors and posttest result is 
high, r = -0.38, for those groups who have not been abl^ 
to correct any responses with the aid of key answers, 
i.e. the FO groups. . ^ / ^ 
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16.7 SUMMARY OF THE RESULTS OF THE MAIN STUDY 



16«7J The Importance of the response requlreit^ent 



The findings of the main study reported in this chapter a- 
gree» In general, fairly well with the hypotheses formula- 
ted concerning the influence of the response requirements 
Rl and R2 upon learning effect, Pl time and learning effi- 
ciency (hypotheses 1:1-3 in. table 7.1 above). It has been 
found _ ^ 

- that learning effect (El) and retention (E2) were some- . 
what greater under the, overt response requirement (Rl) 
than under the covert (R2), and that this difference was 
larger in the lowest grade (64) than in the highest (G6)r 

- that^PI time (PIT) was about tvvice as long under the o- 
vert (Rl) as under --the covert (R2) response requirement, 
and, that this difference was greater in grade 4 than in 
grade 6; 

- that learning efficiency (EFFl) was lower when overt re- 
sponses were required (Rl) than when covert responses 
were required (R2), and that this difference was greater 
in the highest grade than in the lowest. 

It is quite clear, then, that this study gives no support td 
the theory th^t overt response requirement is generally su- 
perior to covert response requirement in programmed instruc- 
tion using linear. Skinner-type programs. This is particu- 
larly obvious if one looks at the results for groups with 
100 per cent key answers, i.e. the normal Skinnerian condi- 
tion, separately. In these groups both learning effect and 
retention .were about the same for the overt as for the co- 
vert response requirenent (El means 12.88 and 12.50 and E2 
means 10.41 and 10.57 for Rl and R2 groups respectively). 
The learning efficiency urfder 100 per cent key answer fre- 
quency v,fas, however, much lower for the overt (Rl) than for 
the covert (R2) response requirement (EFFl means 10.97 and 
14.87, which are significantly different from each other. 



These findings are, on the other hand, in good agreement 
with the theory that the influence of the response require- 
ments Rl and R2 1n programmed Instruction 1s dependent. upon . 
a number of program and student variables. In- particular 
the findings support the hypotheses . . \. 

- that th6 overt response requirement (Rl) has a very 
snmll. positive effect when used with a program requiring 
a smiill amount of response learning^.(t5T. 3.3,2,3» 5.1 
and 7.1 above); and 

- that for a. given program the^overt response requirement 
(Rl) has less positive influence upon learning the ol- 

^ der ancl the better equipped as to background knowledge, 
stucly skills etc. the students are (cf. 3.3.2.2 and 
7.1 above). 

This discussion of the importance of the response require- 
ment in programmed instruction is based on the assumption 
that any response required in a program is relevant 'with 
regard to the criterion test. This issue is further dis- 
cussed below (section 16.7.4). 



16.7.2 The importance of the key answers 



The analyses of data frcMi) the main study reported here give 
some support to the hypotheses formulated regarding the 
influence of key answer frequency upon the various criterion 
variables (cf. 7.2 above). It has been found 

- that learning effect (El) and retention (E2) under the 
overt response requirement (Rl) increased, as expected, 
with decreasing key answer frequency - from 100 to 0 per 
cent - while both effect measures (El and E2) seemed un- 
affected by the variation in key answer frequency when 
covert responses were required (R2)i 

- that the PI time (PIT) increased, as expected, with de- 
crea'^'^ng key answer frequency - from 100 to 0 per cent - 
when overt response was required.(Rl), whereas this va- 
riable (PIT) was not affected by key answer frequency 
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variations when covert responses were required (R2); 

- that the learning efficiency (EFFl) instead of remaining 
unchanged under the overt response requirement (Rl) when 
key answer frequency decreased - as was hypothesized 
(section 7.2 above) seemed to increase, ^^rfrfrFtne" effi- 
ciency (EFFl) under the covert response requirement (R2) 
remained unlnfluenced^by the decrease In key answer fre- 
quency - instead of decreasing as iiypothesized (section 
7.2 above). 

In combination with a demand for overt responses the pro- 
viding of key answers seems to have a predominantly nega- 
tive effect on learning from the program used in the pre- 
sent study% And this Is true in spite of the fact that the 
key answers apparently have been used to a large extent for 
the purpose of control and correction (cf. section 16.5.1 
and tables 16.17-18 above). , ^ 

A negative effect of the availability of key answers upon 
learning from a program seems explainable in view of the 
fact that 100 per cent key answers has led to shorter work- 
ing time (PI time) than 50 or 0 per cent key answers, al- 
though the increased amount of corrections under the 100 
per cent key answers condition must have consumed a certain 
amount of working time. 

One can conclude, therefore, that the availability of 100 
'per cent key answers has caused the students, who were re- 
quired to give overt responses, to devcf'e less time to the 
' content of the program frames. In other words, the effec- 
•tive learning time has been shorter the more key answers 
have been '^prayided in the program, v • ' 

When covert responses were required (R2), the variation in 
key answer frequency seems to have had little .influence on 
learning effect or working time. The most plausible expla- 
nation for this finding is probably that the students have 
' not utilized, the infoHna^ion provided through ;the key an- 
swers. It seems reasonable that the need for control of the 
responses is felt less strongly by the student when the re- 
sponses are not written down or otherwise registered. 
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16.7.3 The combination of overt response requirement and 



100 per cent key answers 



The hypothesis that the Skinnerian combination of overt re- 
sponse requirement (Rl) and 100 per cent key answer frequ- 
ency (Fl) is superior In terms of learnings effect to other 
conditions studied here (section 7.3 above) has not recei- 
ved any support by the results of the main study. In fact, 
both learning effect (El) and retention (E2) under the 
Skinnerian condition (RlFl) were somewhat lower than the 
ayerage effect and retention under the other five condi- 
tions. And the learning efficiency (EFFl) under the Skin- 
^nerian condition of overt response requirement and 100 per 
cent key answers was significantly lower than the average 
efficiency under the Qttier conditions. 

That the Skinnerian model of programmed Instruction, in- 
volving overt response requirement and 100 per cent key 
answers, should be generally superior to alternative forms 
of response requirement and response Information in pro- 
grammed instruction seems, then, very unlikely - particu- 
larly if the efficiency criterion is taken into account. 



16.7,4 Students' errors and corrections in overt 
responding groups 

The number of program response errors made by the students 
increased and the amount of corrections of such errors de- 
creased with decreasing key answer frequency. This seems 
to imply that the available key answers (whether lOD or 50 
per cent) hav^^been utilized by the students for the pur-, 
pose of cofitrol' and subsequent correction of errors. Key 
answers may also have been used for the purpose of a form ^ 
of 'self -prompting', which is often called cheating, i.e. . 
the key answer is read before the student has written down, 
or perhaps even thought out, an answer of his own. 

The correlation between response errors in the program and 
the posttest result was high only for groups with 50 or 0 
per cent key answers (r = -0,77). When IQO per cent key an- 



sv/ers were provided, thts correlation 'was very moderate (r 
.= -0,33). This obviously means that in programmed instruc- 
tion of this kind- vdth the program presented in book form 
- the learning effect cannot be predicted from the number 
of -procrram i-esponse errors made by the students, if key. an- 
swers have been given after each frame. 

The very high correlation (r'>= -0.88) betv/een program re- 
sponse errors made 5y the students and posttest result for 
groups who were provided no key answers (cf. ^able 16.21 
above")' can be regarded as indirect evidence of the relevan- 
ce of the program responses with reference to the; crite- 
rion test (the PUSL test), Holland (90), for example, has 
made .the point that the lack of positive effect of overt 
responding in prograimied 'instruction is often attributable 
to the lack of program response relevance. This is a very 
good point, and it should therefore be observed that the 
lack of positive effect following overt response r^quire- 
.ment - as compared to covert - in the present study can 
hardly be due to such a lack of response relevance. 
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A Synthesis Attempt 



There seem to be three main theories regarding the impor- 
tance of overt response requirement in programmed instruc- 
tion using linear, Skinner-type programs. The first theory, 
represented by, among others, Holland, contends that overt 
response requirement has a general positive influence upon 
learning (90:93ff), and that 

"studies failing to reveal this effect are largely 
explained by failures in meeting the rationale for 
overt responding and by poor experimental procedures. 
(90:101f). 

It is recognised that overt responding is time-consuming, 
but the solution is believed to be found in the develop- 
ment of more expedient modes for the registering of stu- 
^dents' responses (90:102). ^ ' 

/The second type of theory Is represented by Anderson (59). 
He accepts in princtple Holland's theory, but he modifies 
it in admitting that the positive effect of overt respond- 
ing may be greater or smaller due to other factors - prima- 
rily the degree of '^response learning" (59:139ff). The 
learning efficiency problem entailed by the fact that overt 
responding is time-consuming is, in fact, not at all dis- 
cussed by Anderson, although his analysis of the role of 
overt responding is others se quite thorough. 

The third £ype of theory, represented by, among others, 
Leith, dfees not ascribe any generally positive effect upon 
learning to students' overt responding. This kind of theory 
maintains that the effect of overt responding - or demands 
for overt responding - can be positive or negative or non- 
existent, -due to a^-number of other^faotors-in-th€--tota4 — 
situation of programtned instruction. Factors which are be- 
lieved to be particularly important are the degree of re- 
sponse learning, the student's age ^r maturity level and the^ 
student's background knowledge (35), In his discussions of 
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this problem, Leith indicates that he is aware of the lear- 
ning efficiency problem involved, but his empirical studies 
contain no serious attempt at efficiency analysis. 

The results of my own studies and the results of the re- 
search reviewed in chapter 3 above are not consistent with 
Holland s theory concerning overt responding, unless, of 
course, his demands on program design and program length 
are interpreted so literally as to become almost absurd, if 
nis jictum that, in order to demonstrate the supsn"oritv of 
overt responding, "programs must be^Tong enough for sub- 
jects in the covert condition to become careless" (90*93) 
IS very strictly interpreted, then Holland's theory is, -at 
least in this respect, a priori confirmed: if overt respon- 
ding does not show positive effects the program just isn't 
long enough. It also deserves mentioning here that Holland 
discusses in considerable detail the reasons why overt re- 
sponding should be preferred to covert, but in his own ex- 
perimental studies {24 & 25) he only compares overt respon-^ 
se requirement with no response requirement - not with co- 
vert response requirement. This means that he has not real- 
ly attempted to verify empirically his own theory. 

According to Holland it can also be argued that a text must 
be dejpionst rated to be programmed, before studies of respon- 
se requirement etc. are carried out. This implies that the 
responses required of the student in the text must be con- 
'tingent upon the greater part of the text. If this is not 
so, most of the text could be, to use Holland's tertn, " 
blacked out' without this affecting the student's possibi- 
lity of responding correctly. What Holland calls "black- • 
out rati9" (91 & 94) has not been computed for the program 
used in the present study, but since questions in the pro- 
gram concern grammatical examples, it seems obvious that 
the' responses required are contingent upon the text: no 
correct responses could be given without the student having 
studied the examples given in the text. 

As regards the two other types of theory discussed here it 
seems somewhat more difficult to judge their respective me- 
rit. The difference between the two lies mainly in .the fact 
that one - Anderson's - does not allow for the possibility 
that overt response requirement has a negative effect pn 
learning, which the other - Leith's - does. (In a later 



.182 



essay on this subject Anderson (60) has recognised the pos- 
sibility that overt responding can have a negative street 
on learning.) The evaluation of these two theories is furth- 
er complicated by the fact that they are, to a considerable 
extents based on different sets of experimental data, since 
the British studies on which Leith^SB) relies heavily are . 
not at all discussed in the review by Anderson (59). 

As far as the learning effept under different response re- 
quirements is concerned, the results of my own experiment 
seem consistent with Anderson's as^well as with Leith s 
theory regarding the effect of overt responding. Some of 
the studies presented by Leith (35), and also certain other 
results (cf. section 3.3.? above), indicate, however,L that 
the effect of overt response requirement can be negative 
compared to the effect of no response requirement or covert 
response requirement. A study which doesf, not deal with pro- 
grammed instruction in the usual sense, ;{)Ut which is still 
of interest in this context "has been presented by McGuire 
(106), The learning of to61s' names by means of an audio- 
visual device was' stu'die^, using two different speeds of 
stimulus presentation. tt^e higher presentation speed 
the covert response requirement led to higher learning than 
the overt. Apparently, to force the student to give overt 
responses may -interfere with the learning process, if the 
student is still uncertain of what is the correct response.- 

It seems, then, as though the hypothesis that overt respon- 
se requirement can have a negative effect on the learning 
effect In programmed instruction, should not be discarded. 

Markle has proposed in a thought-provoking essay that 

"In most instances the important part of a response is ^ 
what takes place before the overt behavior" (105:193). 

This can be developed into an interesting theory: the co- 
vert response activity is the important part of the stu- 
dent's responding in programmed instruction, and to require 
overt responses will have an effect insofar as it affects 
the probability - positively or negatively - of the occuren- 
ce of the covert response activity. Such a theory is not at 
all inconsistent with Holland's thesis:--s^^ , 
. • "The reason for public (i.e. overt; my comment) answer 
is not theoretical but practical ... pr>|vate (i.e. 
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covert; my comment) Snswering often may change to pri- 
vate omissions" (90:93). . 

A great deat of evidence shows that 'the effect of covert 
response requirement in programmed instruction can vary 
with a number of program and student variables. Which are » 
then, the most important of these variables? 

The ic e 1 e V a n c e of p r o g r a m -r e s p o n - 
s e s IS obviously one of them. Unless there is a definite 
connection between program responses, on the one hand, 
and the content to be learned, on the other, the effect of 
. overt - or any - responding cannot be expected to influence 
the measured outcpme of the learning process. 

?.? length is, as this study has shown, in 

an llkeliness ohe of these factors, although there still 
remains the task, of defining an acceptable measure of pro- 
gram length. ^ 

The g e and the intelligence level 
0 ^.(t h e student also belongs to the group of 
vanahles on which the effect of overt responding depends. 
The rfel^tion between the knowledge, aptitude etc. of the 
student, on the one hand, and the program design, on the 
other, is likely to be of great Ijnportance. This relation 
IS expressed in student-related program variables, here 
called PI variables, such as program difficulty and the 
degree of response learning requtred in a program. The in- 
teraction between student age and the effect of overt re- 
sponse requirement is in that ca^e merely a secondary ef- 
fect; a consequence of the covariation of age with know- 
ledge and aptitude, variables which in their turn affect 
the relative difficulty of a program. In this sense we are 
dealing here with an instance of aptitude-treatment inter- 
action of a fairly precise nature. 

Concerning the importance of key answers in programmed in- 
struction the different positions do not seem to be very 
clearly formulated. Anderson summarizes: 

"..whtle it is generally known that KCR /knowledge of 
correct responses/ facilitates learning, there is good 
reason to doubt that it functions primarily as a re- 
inforcer... KCR may be chiefly ... a source of correc- 



tlve feedback*" (59:151). 

The reinforcing effect, of key answers has been even more 

strongly questioned by Davies: 

"Frequent and apparent success need not by itself be 
Intrinsically reinforcing; the socalled programming 
principle that mere progress through a programme is 
inherently reinforcing is not only poor learning theory 
- it is an abandonment of common sense." (74:206) 

Various reasons why the lack of key answers in a 'program 
should, under certain circumstances, entail bettar learning 
have been presented by Lublin (39:299ff). Generally, the 
evaluation of the effect of key answers in programmed in- 
struction is complicated by the fact that you have to reck- 
on with different types of effect: feedback, reinforcement, 
negative effect on attitudes or motivation; possibilities 
of cheating etc. But our measures of learning can only es- 
timate the net effect of these different influences, some 
of which may even cancel each other. In some cases even 
this net effect of key answer presentation^ on learning Is 
difficult to assess. 

The research review presented here (section 3.5.1 above) 
has demonstrated that the effect of key answers combined 
with a demand for overt responses is dependent upon the 
length of the Instructional program. The nature of -this in- 
teraction isr such that 100 per cent key answers influences 
learning positively in shorter programs (less than 200 fra-^^ 
mes), but has no or even negative effect in longer programs. 
The results of the present study are consistent with this 
finding. It is also noteworthy that the time spent by stu- 
' dents Q,n the program (PI time) is longer when no key answers 
are provided than it is when the key answer frequency is 100 
per cent both in Lublin's study (39) and in my own (sec- 
tion 16.3 above). This seems to indicate that the presenta- 
tion of key answers after each frame made students less in- 
clined to reflect upon the tasks given in the frajnes. 

Anderson, commenting upon Lublin's finding, contends that 
there is a simple explanation of this negative effect, of 
100 per cent key answers: 

"The procedure and program format ... may have permitted 

the students ... to copy the correct answers into the 

blanks" (59:151). 



Such an explanation is, of course* aTv^rays plausible when 
programs are presented tn book form, as in Ijoth Lublin's • 
and rny study, or by means of a simple, not 'cheat-proof 
machine. My analysis of errors made by the students in 
the program (section T6,5J above) has shown, however, that 
the key answers to a great extent have been used for the 
purpose of what Anderson calls "corrective feedback". In 
the groups given 100 per cent key answers on the average 
65 per cent of the errors were- spontaneously corrected by 
the students, whereas the corresponding percentage of error 
corrections in groups provided with no key answers v/as as 
low as 7 per cent! This finding makes it most unlikely tnat 
the students just copied the key answers. After all, /f key 
answers are copied incorrectly, what information could the 
student use to.correct those errors? It. is alsfo worth noting 
that the teachers participating in the study did not ob- 
serve such copying, except on a few occasions- -.^ * 

Lublin's explanation of the negative effect of continuous 
key answer information seems, therefore, more tenable than 
that of Anderson. Lublin maintains that programmed instruc- 
tion with 100 per cent key answer frequency rtiakes the task 
"so unchallenging, or so uninteresting (due to the 100 
per cent. predictability that the right answer would 
inmediately be supplied), that they /the students/ > 
v/ere not attending to the concepts being taught in the J 
frames." (39:300) / 
The ab'sence of key answers, on the other, hand, made / 
"the subjects study each frame more carefully" (39:299/. 

The research review presented in the present study covers 
no studies of the influence of key answers when covert re- 
sponses are required. A study by Kanner & Sulzer (92) is, 
however, of interest in this context, although it does not 
concern programmed instruction directly. These authors 
found that the removal of key answers in a learning situa- 
tion where covert responding, was required affected learning, 
negatively, but not significantly so. 

The conclusions which can be drawn here concerning the ef- 
fect of key answers under the condition of covert response 
requirement must," consequentlyybe based mainly on the 
findings of n\y own experimenJj/These findings indicate that 
the variation in the key ansir^er frequency under the covert 



response requirement (R2i affects neither learn-rng effect 
nor PI time, v^hich means that learning efficiency is not 
affected either. The most reasonable explanation of this 
finding is that the key answers simply have not been used 
fdr the purpose of control to any great extent - or perhaps 
not at all. 

The synthesis attempted here res:ult>i^ then, in the follow- 
ing major conclusions concerning progniBrod instruction, 
using linear, Skinner-type, programs preseftied in book-form: 

- When programs are shor«, less than 100 fkmes, varia- 
tions in response requirement and key ansv^er frequency 
have little influence^ the learning effect^ Consequent- 
ly, the" least time-c0nsumf?tgxEI conditions c|n then be 
expected to lead t9rthe highest learniag^effictency, 

- When programs ar^'fairly long, more than 200 frames, o- 
vert respons^^fequirement with no key answers or covert 
response requirement - with or without key answers - 

can be expectedHo entail better learning than any other 
combination of response requirement and key answer fre- 
quency s^tudied here. Which alternative is the most effi- 
cient will then depend almost exclusively on the *cost' 
. of overt responding in terms of time spent by the stu- 
dent -on the program. 

- Overt response requirement has its greatest positive ef- 
fect when the program demands,. for a given group of stu- 
dents, a high degree of learning of unfamiliar response 
terms or otherwise has a high information level, i.e. is 
difficult to master. 

- Covert response requirement leads to considerably shor- 
ter PI time than overt response requirement, if student- 
constructed responses are demanded. And normal lyv the 
learning effect is almost as great under covert as under 
overt response requirement. If efficiency of learning is 
the primary criterion, covert response requirement can, « 
therefore, be expected to lead to results better than 

- or at least as good as - those reached with overt re- 
sponse requirement. 
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- The Skinnerian combination of a demand for overt re- 
sponses, constructed by the student, and presentinq c 
key answer after every student response cannot in any 
respect be considered generally, i.e. with regard to 
various program types and student categories, superior 
to other possible combinations of response requirement 
and key answer frequency, • . 



Practical Educational Consequences of the Results 



Programmed instruction of the Skinnerian type was original- 
ly developed on the basis of one particular learning theo- 
ry. The acceptability and usefulness of this theory can, no 
doubt, be questioned. And it has been questioned both v^ith 
regard to Its explanation value and its agreement with a- 
vailable experimental data, or rather lack thereof, and for 
its ideological implications ^s a theory of behaviour cont- 
rol and shaping, This is not the place to reopen this «iis-^^ 
cussion. My point here is instead that even if the Skinne- 
rian theory could be accepted, it is doubtful whether laws 
or principles-of practical school instruction can be de- 
ducted from its principles of learning. *The possibilities 

- and also the desirability - of strict control of various 
aspects of learner motivation in a real -life school situa- 
tion are rarely - if ever - comparable to that of a lear- 
ning laboratory situation. 

This problem has been touched upon earlier in this study* 
in chapter 2. It has been emphasized that program charac- 
teristics should not be mixed up and confused with PI func- 
tions, which are dependent upon student behaviour. The 
program characteristic 'presentation of key ansv/ers after 
each frame', for example, cannot, be said to be unequivo- 
cally related to one particular type of student behaviour, 
v/hereas the tenn 'reinforcement' undeniably implies that 
something is happening to - or in the student, although this 
'something* may not be easily observable. 

Decisions concerning the design of instructional programs 

- as welT as those. concerning other' instructional mate- 
rials -must primarily be dealing with program variables, 
i.e. they deal with programmer behaviour rather than v/ith 
student behaviour. Naturally, the wisdom of the deci-sion 

is ultimately judged with reference to some kind of student 
behaviour. But this means only that the behaviour of the 
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student is the dependent variable fay means of which we 
study the effect o-f the independent variable: the pro- 
grammer faehavlour. ^ 

The reinforcement problem provides a good Illustration of 
this point. Immediate and frequent success In working on » 
the learning tasks Is believed to further learning in that 
it functions as a reinforcer of the desired student beha- 
viour. The presentation of key answers after the student 
has responded to, a program frame is assumed to make the 
student experience success and, thus* to reinforce his 
learning of that particular response. This is, for examp- 
le, Skinner's ov/n position {I21:96ff). 

A precise and definite relationship between a particular 
program characteristic - the presentation of key answers 
- and a. student behaviour - experiencing success - is sim- 
ply taken for granted. As a consequence, the positive ef- 
fect of the key ansv^er presentation is also taken for 
granted - and even believed to follow by logical necessity 
front the learning theory in question. To deny - or even 
doubt - the importance of key answers in programmed in- 
struction may then easily be construed as questioning the 
importance of reinforcement to learning, which is quite a 
different matter altogether. The confusion is, of course, 
not diminished if, as often happens, the presentation of 
key answers in an instructional program is labelled 're- 
inforcement* , 

The presentation of key answers in a program can, as was 
pointed out in chapter 17 above. Influence the student^s 
behaviour in several different ways. It may, for example, 
diminish the student's desire to make an effort to solve 
the problems presented to him, which will, in all likell- 
ness, affect his learning negatively. To question the posi- 
tive effect of a program characteristic - the key/ answer 
presentation - doesi therefore, not necessarily iiiean that 
one denies the positive effect of the student behaviour 
supposed to be correlated to the program characteristic, 
namely the student's observing that his ansv/er was- cor- 
rects It simply means that the program characteristic and 
the student behaviour are not believed to be highly corre- 
lated. 
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The response requirement problem can, be looked upon in a " • 
similar vein. The principle that 'learning is doing' .is • 
not necessarily .questioned, because one questions iwhether 
one, in programmed instruction,. should demand overt, stu- ^ 
dent-constructed responses of the type and to the extent 
presupposed in the Sktnnerian PI model. Besides stimula- 
ting, with a higher or lower degree of probabiLity, some 
specific response behaviour (in this case the construction 
and writing down of a response), the response requirement 
may well influence the student's behaviour in other ways 
- which could be desirable or undesirable. In prolonging 
the- time the, student has to spend on the program, the re- 
sponse requirement may, for example, have a negative ef- 
fect upon the student's motivation to work with the pro- 
gram. 

This investigation has studied some alternative response 
'requirements '^nd key answer frequencies in programmed in- 
-structibn using linear programs. On the basis of both pre- 
vious research, and my own experiment I have drawn certain 
conclusions, which were presented in chapter 17 above. I 
want to emphasize that the present study does^not pretend 
to have denwns^rated in exactly what ways these response 
requir^ements'or key ainswer presentations influence the stu- 
dents' behaviour and th^ir learning. What has been studied 
here is primarily^ the n^t effect of different program cha- 
racteristics, ^s this effect appears in measures .of lear- 
ning ef.fect, time spent on the program (PI time) and lear- 
ning efficiency. 

/This net effect of. a given set of program characteristics 
"is, however, of very great practical importance;- And it is . 
a primary objective of this study to try and improve the 
basis for practical educational decisions' concerning pro- 
grammed instruction. Even if our understanding of the me- 
chanisms' of different program designs is much less th^n 
complete, w$ will probably want to make a number of deci- 
• sions regarding what designs to Use in different situations, 
when wa want students to learn by programmed instruction. 
Before making any such decision we should consider very care 
fully all the available data corfcerning the probable net ^ 
effect on Jear^ning and .learning efficiency of each of the 
alternative designs among which we want to choose. 



The conclusions to which the\present investigation has fed, 
and which .have .been presented in Chapter 17, should not be 
regarded as 'definite in any way - conclusions rarely. should.' 
; Sfnce the relevant experience ^nd >e^e^rch results are 
not unanimous and conclusive. These conclusions are, how- 
ever, r3in niy* opinion more reaspnable and Likely to be true 
•than most other possible conclusions, they are also more 
consistent with the available empirical data. These^conclu- 
|ions should, thetefore,.be regarded as a summary of the 
Dasis for decisions regardtng response requirement and key 
answer frequency in progranriied instruction which educatio- 
nal researcfi can offer at present.. 

The practical consequences of theconclusions drawn frpm 
the findings of this stuc(y may not, be altogether self-evi- 
dent. It should, therefore; be useful to try and spefl them 
out. r set. the following consequences as the. most important 
ones^: . * / \ 

- The appltcatfon of principles of learning theory does 
not automatically le'ad to the optimal design of program-' 
med Instruction as regards, response requirement and'key 
answer presentation. 

- The efficiency criterion, i .^yprilmarlly the cost of 
learning In terms of student time, must be considered in 
the choice of programmed instruction model, particularly 
when.tt comes to choosing the-type of response require- 
ment: ^ . . 

- To accept anrl apply /generally, i.e. for all program and 

^ student categories, the Skinnerian demand for overt, stu- 
dent-constructed answers and 100 per cent key answers is 
, not defen'sible, especially not In view of the learning 
efficiency criterion. f, 

- The.less time-consupfiing covert response requirement is 
particularly suitable for programs which, for a given 
group of students, call primarily for associating a pre- 
viously learned response repertoine^^j^ir a limited num- 
^3er of new re^sponse terms -^with n€w concepts, phenomena 

^ etc., i.e. in programs entailing only a small amount of 
response learning. 
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If overt responses are required, the key answer f re- . 
quency should'>preferably be Vess than 100 per cent^ in 
parti cu Tar wheh^he program is fairly Itjng,. I.e. has 
more than .200 .frames or the equivalent thereof. 

As a consumer, of programs *to be presented ir)f book form 
or by means of simple" machines one should not accept da- 
ta regarding errors made> or not made - by students 
going through the program as measure of the program's 
potential learning effect, if these error data have 
been recorded in a situation where 100 per cent key an- 
swers were presented to the^student. J ' 

Finally, ii should be obs^erved that other programmed in- 
struction designs than the traditionaKSkinnerian design 
n}ay make it feasible to construct more flexible types of 
program^" which are more easily adapted to th6 different 
learning tasks involved in various school subjects. 
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Bilaga 1 (s 1) 



Exp PUSL67 
Prov 



KLass : 

Datum ; Grupp : 



A. I var och en av neningama nr 1 - 15 (nedan) elrt'ord understruket. 
Da skall svara pa tv^ fr^or om det understrukna ordetl 

a) Pj? det understr^Doia ordet subjekt eller predlkat i meningen? 
Svara 'genom att sStta kryss i en av mtoma till v&ister on me- 
ningenl 

b) Till vilken ordklass hdr det understru)<na ordet: substantiv, 
proncmen eller verb? 

Svara 'gencm att sMtta kryss i ^ av rutoma till htjiger cm mening- 
en! 

a) ordat Sr » cfrdet hik* till ordklassen 



sub- 
jekt 


predi 
kat 


substan- 
tiv 


pro- 
nomen 


verb 




□ 


1, Har^Jjd spi^inger fort. □ 


□ 


□ 


□ 


□ 


2. R^ven smet in i skogen. □ 


□ 


o 


□ 


□ 


3. Pojkama matte en bund. Q 


□ 


□ 




□ 


^. Dgi hade gatt vilse [D * 


□ 


□ 


* 

□ 


□ 


5. Taget kairoer cm en kvart. Q 


□ 


□ 


□ 


□ 


5. Det bar huset byggdes fiirra liret.*\ □/ 


□ 


D 


□ 


□ 


7. Hon §r 11 Ar gam^nal,. " □ 


□ 


■ □ 


□ 


□ 


8. Ni maste karma i tid. CH 


P 


□ 


□ 


□ 


9. ^ kunde inte sin IMxa.i dag. Q 


□ 


□ 


□ 


□ 


10. I gar kunde ban inte heller sin lilxa. □ 


□ 


□ 


□ 


□ 


li. Snam: konmer taget. ^ E] 


□ . 


□ 


. □ 


' □ 


12. Stugan la£ l^ngt inne i skogen. □ 


□ 


□ 


□ 


o 


13. Nu aker ja^ hem. ' ' C] " 


□ 


□ 


□ 


□ 


14. Flickoma hade vaidigt roligt. □ 


□ 


□ 


□ 


□ 


15. Lisas b^fa vto heter Kias. ' □ 


□ 


□ 



VrtND ikz BIAD FtJRKAN W BLIR TILLSAGD I 



Nam: 
Skola: 



ll var och en av neningama nr 16 - 30 ^^nedan) skall du stryka under 
hela subjektsdelen med ett streck och hela predikatsdelen med tv<i 
strecki 

16. Hunden sk^ler. 

17. &i flicka ramlade. 

Cr 

18. Den svarta katten ftirsvann. 

19. Svenssons bil jiycket gaimai. 

20. Karin tittade ut genom fttnstret och h5rxle inte pa. 

21. Nagra pojkar l^e i skogen, 

22. Tva av dem gjorde upp eld. 

23. De tankte grilla korv. 

2H. En ganroal torr buske fattade eld. 
25. Elden spred sig snabbt. 
25. En av pojkama sprang efter hjMlp. 
27. Snart kom brandkaren. 

28;. Bcide brandsoldatema och pojkama satte igang med slHckiiingen. 
2^. Folk son bodde i n^rheten hjMlpte ocksA till. 
30. Sill slut slMcktes skogsbranden. ' 

VMU INTE BI^ DU BLIR ' TILLSAGD, I 



~ ^il. ICs 3) 

I var och en av'neningama nr 31 US (nedan) sk.all du g6ra fttljande: 

Skriv ett P mitt ovanf6r prediJcatet (ett ord) och ett S mitt ovanfttr 
subjektet (ett ord) \ 
' Stryk"* sedan under hela predikatsdelen med tva streck och hela sub - 
jeJctsdelen ned ett sti>eck! 

-> • 

31. En vit katt sprang farbi, , 

32. Den dSr honden skcLLler ald!rig, 

33. En TGd bil stcir pi garden, 

34. Den lilla pojken vinkade ivrigrt, 

35. MSnen lya^ an natten, 

36. En flicka fr^ JiinkOping vann tSvlingen, 

37. N%ra elever pratade hela tiden, 

38. Det sista t5get snart. 

39. Bam under 6 5r Sk.er gratis. * _ * 

40. Alia bAtama i hanmen seglade ut. 

41. D^r borta komner en bil. 

42 . Cm dagen sever alia f laddermtiss . 

43. Den h^ meningen sv^. ^ 

44. Zebrwi Sir ett ra^digt djur. 

45. Den har svarta och vita render. 

,^ » 
^LUT PA PROVET 
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* " ' • . .-..^ Bilaga 2 (s 1) 

Statais Ffirstiksskola, Lijik5ping ' ; ? Exp PUSL67 

- . - * Instr pl/al 



INSTRUKTION FQR PROVmARE/ARBCTSLEEARE VIP EXP PUSL67 * 
(pi: provledare, al: arbet sled are) 

Lektion 1; DBA-testnin^ , 

PI logger fare leJctionens start ut provblankett DBA: 2 (nctsatser) pd alia elev- 
platser. (Se till att blankettema l^s med r^tt sida upp.) 

NSr elevema intagit sina platser sSlger pi: , 
"Ut papper>et, son ni har fAtt, ligga b^en och lyssna pA mig. .Den h&r 
^ klassen pka under don narmaste veckoma vara med i ett ftirstik. Det ftirsG- 
ket gSr ut pA att prttva olika satt att 1^ sig saker pA egen hand. Innan 
VI stri-tar f tirstiket vill vi veta hur 'dukrtiga ni ^r pA att Ittsa yi^sa sor- 
ters cr.> lifter. DMrfOr ska ni i dag gA igenom tre olika prov - tvA prov* 
den hi&v tunmen och ett prov en annan tirnne. \ 
Jag kcfimer att tala om precis hur ni ska gGm - n^ ni ska bdrja och sluta, 
nar ni ska v^nda blad ogv. G6v som jag s^ger -^men gttr ingenting annati 
^ Ni ska arbata snabbt. Men slarva inte/Och var och en n^ste arbeta helt 
egen hand utan att kika pA sina kamrater. — " 
Nu ska n^ skriva namn, f&lelsedatum, skola 6ch Arskurs-nedtill det oaD- 
per, som ni har fAtt." . • ^ ^ 

war alia hai^ fyllt i dessa- saker, forts^tter pi: : 
"Ni har nu frtamfor er anvisningama till ett prov som heter *Motsatser«. 
FGlj med i anvisningama, nSr jag g^ igenom dem." 
PI mser anvisningama httgt. Uppehdll giires, att elev^ifna kan f ylla -i 
svaren till dvnijTgsexemplen. DSrefter kontrolleras svaren till tivningsexemplen 
genom fnagor till nigon eller nigra elever. Se till att alia elever klart 
far sig^vad sonKSr ratt svar pa resp. tivningsexempel. 
Facit: Ovn.e^ DBA: 2 uppg. .2345 
svar .3142 

D&>efter fragar pi: 

"Ar det nagon^som inte fttrstatt, hur ni ska Ittsa sina har uppgifter?" 

Eventuella fragor besvaras i enlighet ;ned anvisningama Ct.ex.: "Se efter 
vilket av de fy3;^a numremde orden son Sr motsatsen till det fttrsta orde't. Skriv 
numret pa detta Ord, 1, 2, 3, eller 4 i svarsrutan.") Sedan sMger pi: 

"Nu ska vi ga ttver till sjaiva provet. Eet bestir av sanna slags uppgifter 
som ovrtingsexwlen. VMnd blad. — Btirja." 

Efter exak\lO minuter sager pi: 

"Tiden slut. L^gg ifr^ er pennoma." 

PI liter samla in blankettema till DfiA:2 och dela ut blanketter till DBA:3 
MBokstavsgrupper). NSr ^etta Sr klart s^ger pi: 

» "Ni har nu^framfor er anvisningama till ett prov son heter ' Bokstavsgnip- 
per'. F51j med i anvisningama, nMr jag gir igenom dem." 
^Pl laser anvisningama httgt. Uppehill gttrs, si att elevema kan fylla i ^ 
svaren till 5vningsexeinplen. DSrefter kontcolleras svaren till tivningsexeirplen 
g^nom fragor till nigon eller nigr^^elever. Se till att alia elever fir JOart 
far sig, vad som rMtt svar pa resp. avningsexempel ^ 
Facit: ^ivn.ex« DBA:3 uppg. 2 3 4 5 6 7 
svar 1 1 , 3 2 1 3 

DMref ter fragar 'pi : 

[' Ar det nigon son inte har farstatt,' hur ni ska lasa sina hMr uppgifter?" 

Eventuella fragor besvaras i eniighet med anvisningama (t ex: "Det gailer . 
att se efter, vilken bokstavsgrupp, som skiljer sig frin dom avriga. ^^um- 
ret pa den gruppen ska man skriva i svarsrutan"). Sedan sMger pi: \ 

"Nu ska vi ga aver till sjSlva provet. Det bestir av saima slags upj^ifter " ' 
som avningsexempleh. — V^nd blad. — Bai^ja." 



Bil. 2 (s 2) 

Ef ter "exakt 15 minuter Sclger pi : ' * 
"Tiden slut. Lagg ifrSn er pennoma»" 
Darefter samlas provblankettema in. 

Lektion 2: Kunskapsprov. i sprikiara (ftirtest) 

PI delar ut provh^ftena (markta "Exp PUSL67 , Prov" i iivre htigra htimet) till e- 
le^^ema och i^ter dessa fylla i namn, .klass, skola och datum tiverst pS ftirsta 
sidan- i Mftet (i uti ynmi et fiJr gruppnunmer - "Grupp: ~ skall ingenting 

s>xivt^.). NMr alia eleverna har gjort detta Sciger pi: 

"LMgg ifr^ er pennarna, och lyssna noga. — Ni brukar ju ofta fA prov fiir 
att ni ska fA visa vad ni har ISrt er i skolan. - Men det Mr Sr inte nA- 
got s^t prov. Det hSr Ji^tet inhehciller ocksA uppgifter pA s&nt earn ni 
annu inte har ISrt i skolan. Ni ska fl ISra 'fer dom sakema 'ISngre 
fram. hSen nu vill vi' veta, om det Sr -r^igra jav er, som kan' det andA — fast 
ni iiite har f Att lara er det i skolan. — l^i ska alltsA inte bli ledsna, 
on det ar nAgot ni inte ftirstSr alls. Gbr si gott ni kair, si blir,det nog 
r bra, — De uppgifter ni absolut inte kan Itisa, fAr ni hcqjpa tiver, — 
Nu ska jag tala om hur ni ska giSra. ^ 
Det ar tre blad i haftet. Vi ska arbeta med ett blad i taget oph ni fSr 
aldrig vanda blad fiirran jag sager till. — 

PA fiirsta sidan har vi A. t3ver>st pA sidan^stAr det hur man ska liJsa upp- 

giftema (pi hAller upp haftet och pekar). — . 

Nu IcLser vi doin anvisningama'. -Ftil j vne4 i hHt-tet allihop.'' 

Delgrov^A; „ . ^ 

PI JMser ordagrant upp anvisningama till delprov A ur haf "^et och' eager se- 
dan; . . * . 
'»Kom il^ alltsA, att ni ska s^tta kryss ;pA tvA stailen vid var3e mening: 

dels i en av de bada rutoma till vanst^r (hAll upg haftet och peka), dels 
, i en av de tre rutoma till htiger (peka). Ta linjalen till hjaip, sA ni 

hAller er pA ratt rad." 

PI visar eleverna hur de ska lagga linjalen tvars iiver sidan, strax under 
resp. rad med mening och tillhiirande svarsrutor.< Sedan sager pl^. ' 
"Ta upp pennoma. - Om ni gliinnier bort, hur ni skulle gtira, sA las anvis- 
ningama tiverst pa sidan 6n gAng till. — Nu f5r ni bttrja." 
Efter exakt 6 minuter^ sager pi: . ^ 

Lagg ifran er pennoma och vancj blad. "Har har vi prov B. — Nu laser vi an- 
' visnihgama fiverst pA sidan. - Ftilj med i haftet allihop." 

2elprov_B: 

PI laser ordagrant upp anvisningaml till delprov B ur haftet och sager 

"Ni ska alltsA stryka under subjektsdelen med ett streck och predikatsdelen 
med>tva streck. Och ni ska gtira sA med varje mening. — Anvand linjal^ nar 
ni stryker under. — Ta upp pennoma. — Om ni gliSnmer bort, hur ni skulle 
g5ra, sA 3.as anvisningama tiverst pA sidan en gAng tili. — Nu f Ar ni 
b5r3a." 

Efter exakt 6 minuter^ sager pi: 
"Lagg ifran er pennoma och v^nd blad. — " 

— ;Dm s a m t 1 i g a elever uppenbart slutar arbeta med pro- 
vet, fir pi avbryta provet 3 0 s $ k efter det han konstaterat, 
att sa ar fallet. I sAdant fal^ skall pl^ocksA notera, hur mycket prov- 
tiden darigencm ftirkortats. 



ERLC 



209 



. • ' . ' to. 2 (s 3) 

^Delgrov C : . r - ^ 

/tu sSger vidare: ' ' 

jj..,»^Kar har vi prov C. — Nu laser yi anvlsidngarna tiverst t>i sidan. - Ftili ■ 
i 1 haftet allihop." ^ - 

t ?^l^sen ordagrant upp anvisningarm till delprov CTur h^tet och s^er se- 

*'"Kom ou ih^ att gtira allt det som st^ i anvisningama. — Skriv P ovanftir 
^ I^^edaJcatet och S ovanftir subjelctet. Stryk sen under 'predikatsdelen med tvi 
, streck^cxph subjektsdelen med ett sti^. - Och gtir si med varje maninR. — 
\ AhN^ lineal ni stryker under. Ta upp pennoma. — On ni gldi^ 
, bort, hur ni- skulle gtira, -sA ISs anvisningania tiverst p5 sidan en gins 
^ till. — Nu f ni bOrja," . ^ s >o . 

Efter e xu,a k t 1 0--' m i n u t e r^ sSger pi: 

"Lagg ifrtOs-er pennoma. Nu ar provet slut. — Kontrollera nti att ni fyllt 
* ^^b&^keif"^^'-^^^^ och datum p5 fiirsta sidan. lAt sedan prwhSftet ligga 
PI samlar in provhaftena^vai anVSnda son ev, oanv^nda, och behAller dem 
1 sitt ftirvar, tills han personligen^kan tiveriamna materialet till ftirsttks- 
ledaren. ^ \ , 

^ Lektion 3; Inskolning i arbetssattet f tir resp grupper (1-6) 
Fbre lektionen'skall al ' 

1) kontojllera att material erl^its fdr r^tt grupp ( jfr arbetsordning och ut- 
rustn.anvisn.) o- rr j 

2 ) kontrollera att banc^pelare finns pt platf och ar i anvandbart skick 

^udbSJd^f lampligt att al i /Orvag har lyssnat p5 instruktion^ 

• 3) skicka den egna klassens elever - utrustade med gnippnuirmerlappair - till 
resp salar ijfr arbetsordning samt ftirteckn. Over .gruppindelning) . 
NSr elevenja anl^t till salen vid lektionens bttrjan skall al kontrollera 

1) . att end^st elever mfed r^tt gruppnuimer finns i solen 

2) att det -har aniant elever fr^ samtliga de klassS, stm 6nl. arbetsan^ninc- 
en skall^ samordnas under lektion 3, . . .j 

Nar ma" elever har den material som behtivs ftir lektiOnen vA sina bankar (ifr 
utrustningsanvisn.) .lAter al dem fyila i namn, klass, skola och datum pi ftirsta 
sidan 1 arbetshaftet. .t^ 
Sedan sag^r al: 

^ "I dag ska vi bSrja ett'ftirsttk, dar ni s}<S fl arbeta mit pA egen hand med 

en rad arbetshaften. Ni ska ftirst U lara er hur ni ska arbeta. Men det ar 

xnte 3 ag som ska tala cm det ftir er. - Alia anvisningar finns p5 det har 

Ijudbandet. Nar jag satter ig^ bandspelaren, ska ni ftilja med noga alli^ 

hop 1 ^arbetshaftet och gtira precis som det sSgs. Om' " ni har nigot att fra- 

S?* ^zJ^ ^ ^f^"'^^ ^^^^^ ^ spelat igenom bandet. — Ap alia klara? 
— Da satter vi ig^." 

Al startar bandspelaren. 

Qm al.bliiv t v u n g e n att svara p5 n^on frSga eller p5 annat satt in- 
gripa, SKall bandspelaren stoppas tills saken ar uppklarad. 

^H^r«^\^-^!J^^u"^^ uppnwun^en: "Nu f^ ni fortsatta sjaiva med det .har haf- 
• VL ^ ^ besvaa^a fragor om arbetssattet i enlighet med 

ojistniJctionerna pA Ijudbandet och den s.k. kom-ihAg<lappen. Al:s svar bt5r om 
majligt inskrankas till uppnvaningen "L^s pi kcm-ihSg^lappen" eller uppiasning 

^) Chi s a m t 1 i g'a elever u p p e .n b asr t slut.^ arbeta med pro- 
.vet, f|r pi avbryta provet 3 0 s e k efter det han konstaterat, att 
si ar fallet- I sAdant fall skall pi ocksi note^ia, hur myck^t provtiden 
darigenom f^Jrkortats. ' * ' ^•r ^ 



Bil. 2 Cs ^) 



av vad scan stScc pi den lappen. Arbetstiden behtiver inte nateras under denna lek- 
tion. Nar alia elever,ar f^rdiga med hMftet,^avbryts verksaniheten CMven arbetet 
med rajcneuppsiftema) . DfiLrefter insamlas all inaterial. 
OBS: I h^tet "Fttrprogram" (version 1 och 2) fttrekcamnsr det, att det r^lfta sva- 
ret lyser igencm, sA att det kan ISsas utan att man vender blad. Denna 
hrist bar elininerats i huvudprogc^nmets hSften. Om n^on elev under lek- 
ticn 3 pApek^ att svaren lyser igericmi, skall al upplysa hela gruppen 
cm att det inte konmer att vara si i de fttljande hSftena. , « 

Lek t^coa ^-9; Genccng^ av resp. haften i fflpogramnet "Den enkla meningens huvud- 
delaF ^ " , ' 

Fbre varje leJction skall al 

1) kcintrollex>a att mterial erhillits fttr r^tt grupper ( jfr arbetsordijing och\ 
utrustn.anvisn. ) 

2) skicka iv^g de elever i den egna klassen,' son akall till en annan sal - ut- 
iMStade med gruppnuirmerlappar (jfr arbetsordning) . 

Nar alia elever fitt den nata?ial son behtiys f6r lektionen pA sina bSnkar . 
(jfr utrustn.anvisn.), lAter al dem fylla i namn, klass, skola och datum pi ar~ 
betshaftets fttrsta sida sanrt namn och klass pi bladet med r^lkneuppgifter. Sedan 
sager al: . , 

"I dag ska ni arbeta med det hSr haftet. Var och en ska arbeta precis pa det 
satt han eller hon har fitt lara sig.. Ftir sakerhets skull har ni fdtt var • ■ 
^r^in kom-ihig-lapp. Pi dpn stir det hur ni ska arbeta. $aiihop t/st i- 
genom kont-iSig- lappen . CPaus ftir lasningen.) Ni fir gama lasa pi lappen om 
igen nedan ni ^betar. Om ni andi m i s t e friga om nigot, sa tack upp 
handen och vanta tyst och stilla, tills' jag kaiiner till er. Nar ni ar far-- 
diga med hela haftet fortsatter ni med rakneuppgiftema. -Kbm ihig en sak: 
Det har ar inget pirov och absolut ingen hasti^etstaviing. Det ar inte vik- • 
tigt att bli fardig si fort scm mttjligt, men det ar viktigt att arbetai or- 
dentligt och noggrant. Nu fir ni btirja." - ; ■ ' 

Al antec knar klockslag (avr. t. hel minut) , nar .arbetet 
pibdrjats . (Utryiime harftir finns pi bladet "l^trustningsanvisningar") . 
Under arbetets ging fir al endast ge vagledning ang. arbetssattet , Detta sker 
genom hanvisning till kom-ihig-lappen samt ev. uppiasning ftir enskild elev av 
vad scm s'tir dar. (Kom ihag att alia elever inte har samma instruktion.) Al gir 
ront och antecknar k'l c^c k s 1 a g (avr. t. hel minut), nar v^je 
elev avslutat arbetet med haftet. Nar alia elever ar klara med arbets- 
haftet, kan verksamheten avhrytas. Al liter de elever, son bdirj^t med rakneupp- 
gifter, markera pi uppgif tsbladet , hur^ lingt de^hunnit. Darefter insam- 
las all m a t e .r 1 'a 1 . Bladen med r^euppgifter skail al behilla och 
utdetla pi nytt nasta ftirstiksldktion (torn lektion 9). Elevema' f ir alltsi nytt 
•'blad med rakneuppgifter ftirst nar de gjort fardigt uppgiftema pi det ftoa. 
OBS I Al miste givetvis\se tilj., att varje elev fir hafte resp. kom-ihig-lapp ftir 
den grupp sOm eleven tillhtir. 



'Lektion 10: Kunskafisprov Y sprikiara (eftertest 1) 

PI delar ut provhaftena till eleverna och liter dessa fylla i namn, klass, sko- 
la, datum och gruppnunmer tiverst pi ftirsta sidan i haftet. Nar alia elever har 
gjort detta sager pi; 

"Lagg ifran er pennoma och h6r upp. — I dag ska ni fi visa vad ni har lart 
er av de haften ni har arbetat med de senaste veckoma. Det har provhaftet 
bestir av tre blad. Vi ska arbeta med ett blad i tag^t och ni fir aldrig 
vanda blad ftirran jag sager till. *Pa ftirsta^sidah har vi prov A. Upptill pi 
sidan stir det hur man ska Itisa uppgiftema (pi hiller upp haftet och pe- . 
kar). — Nu laser vi dom anvisningamajl Fttlj med i haftet allihop." 
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Bil. 2 (s 5) 

Delproy_A; 

^^^^^^^ till delprov A ur haftet och sSger sedan: 
_ "Kbm ih^, alltsi, att ni ska sStta kryss tv^ stailen vid varje merrmg: 
' dels 1 en av de bdda rutoma'till vanster (pi holler upp hSftet och pekar), 
'- 1 en av de -tre rutoma till hiSeer (pi oekar). — Ta linialen till 

: hj^p, sa ni-hAller er pd r^tt rad.l' ^ 
Pl;^visar elevema hur de ska la^ linjalen tvte iJv6r sidan, strax under resp. 
rad med mening och tillhiirande svarsrutor. Sedan s^iger pi: 

"Ta upp pennorna. - Cin ni gliJmner bort, hur ni skulle g6ra, sA 1^ anvisning- 
ama ttverst.pA sidan en till. — Nu ni bdrja" ' 
Efter exakt 6 minuter s^er pi: 

ifr^ er pennorna och vSnd blad. HSr har vi prov B. — Nu laser vi an- 
visningaJTC ttverst pd sidan, FdJlj med i hSftet allihop." 

Del^ov^B: • p 

PI laser ordagrant upp anvisningama till delprov Burihaftet och sSger sedan: 
"Ni ska aXltsA stryka under subjektsdelen med ett streck och predikatsdelen 
med tvd streck. Och ni ska gdira sA med varje mening. — Anv^nd linjal, nSr 
C> ni stryker under, -« Ta upp pennorna. — On ni elttimer bort, hur ni skulle 

^ anvisningama fjverst pd sidan en ging till. — Nu f ^ ni btiria." 
^ Efter exakt 6 minuter s.ager pi : • 

"LSi^g ifrSn er pennorrk och vand blad.— Har har vi prov C. — hftj laser vi 
anvisningama ttverst sidanrFttlj ned i haftet allihop." 

Delprov C: 

PI laser ordagrant upp anvisningama till delprov C ur haftet och sager sedan: 
"Kbm nu ihig att gtira allt det som st^ i anvisningama. — Skriv P ovanfGr 
predikatet och S ovanftir subiektet: Stryk sen under predikatsdatm med tvA 
' streck och subiektsdelen med ett streck. — Och Rttr^ med varje n^ning. — 
• Anvand linjal, nar ni stryker under. — Ta upp pennorna. — Om ni gltinmer 
bort hur ni skulle etira, sA las anvisningama iiverst pd sidan en till 
— Nu f^ ni Jbttria. 
Efter , exakt 10 minuter \ uger pi: 

"Lagg ifr^ er^ pennorna . Nu ar'provet slut. ~ Kbntrollera nu att ni fvllt i 
namn, klass, skola, datum och eruppnurnmer pd fiirsta sidan. Lit sedan prov- 
haftet ligga pA banken." » 
^ PI samlar in provhaftena, sAvai anvanda som ev. oanvanda, och ser sedan till, 
att all raterial -Ateriamnas till f ttrsiSksledaren. 

Lektionll: Kunskapsprov i sprdkiara (eftertest 2) 

PI delar ut provhaftena till elevema och later dessa fylla i najm, klass, sko- 
la, och datum sairrt gruppnummer (om upp^iften ar tillganglig) . Nar alia elever 
gjort detta sager pi: , 

"I^gg ifrdn er pennorna och httr upp, — Ni kcrnner saJcert ihag, att ni ftir 
nagra veckor sedan gick igenon en del arbetshaften i sprakiara. De handlade 
mest om subjekt och predikat och liknande saker. — Nu ska vi se vad ni 
mmns av det ni l^io er da. Det har provhaftet be^tar av tre blad. Vi ska 
arbeta med ett blad i taget och ni f^ aldrig vanda blad fGrran jag sa^er 
till. PA farsta sidan har vi prov A. Upptill pA sidan stAr det hur man ska 
iusa uppgifterna (pi haller upp haftet och pekar). — Nu laser vi dom anvis- 
rdhgama. FBlj med i haftet allihop." 

Del^;ov_A: ^ <• z 

PI ikser ordagrwit upp anvisaingarna till delprov A ur haftet "och sMger sedan: 
"Kom ihig alltsA, att ni ska satta kryss pk twA stailen vid varje mening i 
dels i en av de bdda rutoma till vanster (pi haller upp haftet och pekar). 
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dels i en av de'Nxe rutctma till htigePsPCpl pekar). Ta linjalen till 

hjaip,^sA ni h^er er pA ratt rad." 
PI visar elevema hur de ska ISgga linjalen tvar^ tiver sidan, strax under resp. » 
rad roed rrening dch tillhfirande svarsrutcr. Sedan Sclger pi: . " 

"Ta upp pennoma. - Om ni gliinmer Ixjrt, iiur ni skulle gttr>a, gI las anvis- 

■ ningama tiverst pA sidan en gSng till." — Nu ni bfirja." 
Efter exakt 6 minuter sSger pi: , ' , 

"L^gg ifr^ er pennoma och vSnd blad. HSr har vi nnsv B. — Nu laser vi an- 
visningarna tiverst p^ sidan. Fttlj med i haftet alSihop.tJ 

'Del^rov^B: ' 

PI laser oixiagrant upp anvisningarna till delprov B ur haftet och^sSg^ sedan: 
"Ni ska alltsA stryka under subjektsdelen med ett streck och predikatsdelen 
mad tvA streck. Och ni ska g6ra si med varje roening. — Anvand linjal, n^r 
ni stxyker under. — Ta upp pennoma. — On ni gittnmer bcrt, hur ni skull6 
gtira, sA las anvisningarna .tiverst pA sidan en gSng till. — Nu fAr iii 

Efter' 'exakt 6 minuter sager. pi: . .® 
"Lagg ifr^ er pennoma och vand blad. — Har har vi prov C. — Nu laser vi 
anvisningarna tiverst pA sidan. F51j med i haftet allihop." 

Deljgrov: % 

PI laser crdagrant upp ^visnihgarna till delprov C ur haftet. och sSger sedan: 
"Kom nu ihAg att fetira allt det som stAr i anvisningarna. — Skriv P ovanfdr, 
piC^ikatet och S ovanfttr subjektet. Stryk sen under predikatsdelen med tvA 
struck och subjektsdelen med ett streck. — Och gdr sA med varje mening. — 
Anvand linjal, nar ni strykerounder. — Ta upp pennoma. Om ni glttmner 
bort hur ni skulle gtira, sa las anvisningarna tiverst pA sidan en gang till; 
— Nil ni.b6rja." 

Efter exakt 10 minuter sager pit 

"Lagg ifriSn er pennoma. Nu ar'provet slut. — KbntroUera nu att ni fyllt i 
naran, klass, skola, datum och gruppnummer fttrsta sidan. LAt sedan pr9y- 
haftet ligga pA banken." ^ 

PI samlar in provhaftena, sAvai anvSnda som ev. canvanda, och ser sedan till, 

att all material dteriamnas till ftirstiksledaren. 
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Staterjs Farstiks^sJsoi^ Bilaga 3 (s 1) 

Linkttping Exp PUSL67 

^ Instr. fttrprogram 

INSTRUKTION VID BJUI^ING AV, FORPROGRAM I EXP PUSL67 

G r u p p . l (krav yttre svar, IQO % facitsvar) 

ifr^ er pennoma och lAt hSftena ligga pA banken. — L/ssna nu noga. — 

Ni.ska under de n^rrraste veckoma fd prfiva pA en ny typ av ,arbets&vningar i 
sprAkMra. Det ar arbetsh^ften, som ni ska fl arbeta med-'Kelt''^ egen hand. 
Alia elever som anvSnder de Mr hSftena f&r sajine arbetsuppgifter. Men olika 
gprupper av elever ska arbeta pA oliJ<a satt med uppgi-f tema. Sen vi se vil- 
ken grupp som lyckas 'bSst . 

Ni som sitter hSr tillhi5ir allihop sairma grupp - grupp 1. Ni ska allihop arbeta 
pA samma satt. I dag ska ni f A ISra er hur ni ska arbeta. Titta nu det ar- 
betshSfte som ni bar fdtt. Pd fiirsta sidan har ni fyllt i namn ocH en del andra 
saker. Nu vender ni blad cch tittar pA nSsta sida. — Varje fiida i hMftet har tvA 
uppgiftsrutCQp, precis som den har sidan. Ni ska btirja med ttversta rutan pA ftirs- 
ta sidan. Sen ska ni forts^tta med iJversta rutan pA nSsta sida, osv till sista 
sidan i hSftet. N^r ni har gitt.igencm den iJversta 'rutan pd' varje sida, dA btir- 
jar ni fv&n bOrjan av li^ftet igen. Men dd med den nedersta rutan. Sa fortsStter 
ni med den nedersta rutan varje, sida till slutet av h^tet. 

* . ' ' \ .. 

Hi skf>. alltiid Wrja med att ISsa noga allt son st5r i rutan, I en del rutor 
films en fr^a och pl^ts ftir^ett svar. DMr ska ni skriva svaret pel fi^gan. I 
andra rutor ar det nAgot som fattas. ska ni fylla i det ord eller dom ord 
son fattas. NSr ni ar fMrdiga medien r\ita, ska ni kontrollera att ni har gjort 
ratt. Det. ratta svgoret - eller det scm fattades - st5r i narginalen till vans- 
ter pA nasta sida. Nu tycker ni kanske att iian'kan titta pA det ratta svaret 
ftirst odi sen skriya av det. Visst kan man det,* men dA IMr nvan sig nog inte 
mycket.— Ska man lara sig det har ordentligt, n^te man f first tSnka efter och 
svar^a sjaiv och sen kontrollera att nan gjort ratt. — 

Nu ska vi gA igencm en dei av haftet fillsantnans , sA ni sSkert f firsts hur ni 
»ska arbeta. Ta pennan i handen och ftilj med i haftet hela. tiden. — Uis texten 
tyst fiSr er sjaiya samtidigt som jag laser upp den. — ' , 

Vi laser ruta l.^Dar star? "Substantiv — Titta noga p^iordst. Se efter hur det 
stavas. — Hur mSnga bokstaver ar det i ordet substantiv?' Svar: ..." Ja, hur 
m^a ar det? R^kna dan och skriv svaret. Nu vander vi blad. — Har ni 
svarat ratt? Se efter i n^ginalen till vanster. — Dar st5r t i o . Det be- 
tyder att ordet substantiv har tio bokstaver. — 

Vi. laser ruta 2^ Dar star: "I en bok firms nassor^'av ord. Det finns olika slags 
ord, olika ordklasser. Ett slags ord kallas substantiv, Substantiv ar namn pA 
en ..." Ja, dar fattas ett orti. Vilket ord fattas pA sista raden? — Tank efter 
vad det ar och skriv dit det ordet. — vand blad. Se efter i narginalen vilket 
ord det var som fattades i fiJrra rutan. — Om ni gjort fel, sa far ni gaAa ga 
tillbaka till fi5i?ra rutan och andra sA att det blir r^tt. — 
Vi^ laser ruta 3. Dar stAr: "Det ar bara en sarskiid sorts ord som kallas sub- 
stantiv. Ordet penna ar ett substantiv, men ordQt skriva ar inte nagot substdn- 
tiv. Vilket ord ar ett substantiv, penna eller skriva? Svar: ..." — Tank efter 
vilket det ^ och skriv svaret. vand blad. — Har ni svarat ratt? Se efter i 
marginalen till vanster. — Dar star "penna". Ordet penna ar ailtsA ett suljstan- 
tiv. — . ' , 

yi laser ruta 4. Dar star: "Ordet tval ar ett subst<^tiv, men ordet tvatta ar 
inte nAgot substantiv. ^ Vilket ord ar ett substantiv, tvAl eller tvatta? Svar:.." 
— Tank efter \?il3cet det ar och si<riv svaret. — Vand blad. Se efter i rrergina- 
len om ni har svarat ratt. — Dar stAi» "tval". Ratt svar ^ alltsa "tval". — - 
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■■■ ^ ' ^. . 

Vi l^ser ruta 5. DSr st^u "Ordet boll ^ ett substantiv, nen ordet kasta Mr 
inte Mgot substantiv. Vilket ord ar eli> spbstantiv, boll eller kasta? Svar:... 

Tank efter vilket det Mr och skriv svaret. — VSnd blad. Se efter cm ni bar 
svarat rSltt. — Svaret ar "boll". iDet stir i marginalen till v&ister. — 
Vi laser ruta 6. DSr stir: ^»Har ar tvA ord: Pehna, skriva. Den ena ordet ar 
namn en sak, 30m du ofta hiller i. Det andra ordet ar inte namn pa en sak, 
det ar nigot man kan gOra. Vilket ord ar nantn pi en sak? Svar: ..." Tank efter 
vilket det ar och skriv svaret. — vand blad. — Har ni svarat ratt? — Se, efter 
i fiBTginalen. — Svaret ar "penna". — ^ r^^^ ^ y ' 

Vi laser ruta 7. Dar stir; "Har konmer i^/i ord: kasta boll. Det ena ordet ar 
n^ pi en sak, son man kan leka med. Det andra ordet ar inte r«inn pi' en sak, 
det ar ju nigot man gOr. Vilket ord ar namn pi en sak? Svar: ..." — Tank ^ter 
vilket det ar och skriv svaret. — ^Vand'blad. — Har ni svarat ratt? — Se efter 
i marginalen till vanster. 5 , ^ ^ ^ ^ 

Vi laser ruta 8.* Dar stir:' "Har* koimier tvi ord: bd<, lasa. Det ena ordet ar nantfi 
pi en sak, men det ar inte det andra. Vilket ord ar namn pi en sak? Svar: ... 

Xank efter vilket det ar octi skriv svaret. — Vand blad. — Se efter cm ni 

Kar svarat r^tt. — . ' . -u ^ ^• 

Vi laser ruta 9. Dar stir: "Alia ord, som ar namn pi ^aker, kallas substantiv. . 

Har konroer.tvi ord: spade, grava. Vilket av orden ar ett substantiv? Svar: ... 

— Tank efter vilket det ar och skriv svaret. — Vand blad. — Har ni svarat^ 
ratt? ~r Se efter i narginalen. — ' , 

Vi laser r\ita 10. DSr stir: "Ai:^ namn pi saker ar' substantiv. HSr koniner tvi 
ord: tvatta, tvil. Vilket av dem ar ett substantiv? Svar: ..." — Tank efter . 
vilket det ar och skriv svaret. — Vand blad och se efter om ni bar svarat ratt. 

— Det ratta svarat stir i marginalen tillL vanster. ■ / 

ligg ifrin er pennoma, — Nu vet ni sakett allihop hur ni ska arbeta med sina 
har haften. Ni ska strax fi fortsatta pi fegen hand med det har haftet. Ftir sa- 
kerhets skull har var oc^ en fitt en kom-tMg-lapp. Dar stir det vad ni sar- 
skilt ska tanka pi, nar ni arbetar. LSs igdnom den lappen cm ni ar osakra. Ni 
fir lasa den nar ni vill och hur ofta ni vill. 

Om ni, upptacker att ni svarat fel pi Mgoa fr>iga, si fir ni gama andra. ert svar^ 
si det blir ratt. " . . 

Nar ni kcmner till Svejrsta rutan pi sista sidan, kan di ihig att gi tillhaka 
' till b5rjan av haftet och fortsatra med den nedersta rutan pi varje sida. 
Nar ni ar fardiga med dit har haftet, tar ni itu med de rakneuppgifter son ni 
har fitt. — 

^ Nu f ir ni fortsatta sjaiva med det har haftet." 
SUUT PA BANDET 
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, EXCERPT FROH THE INSTRUCTIONAL PROGRAM 
Statens Fbrsbksskola ^ ' 

E.P.PUSL67 
^ ArbetshSfte 

\ / 

DEN ENKIA MENINGENS HUVUDDELAR 

V e r s 1 on 1 (fbr grupp nr 1 och 4) 

H S f t e 1 

• ■ ' . 

Klass: 

Sko1a: „ ^ 

c Datum: 

Arbetet nied detta hSfte avslutat klockan 

5^ 1^ V" 



NoteJ.: In order to prevent readthg of key answers through the semi- 

transparent paper, every sheet of paper 1n the Droqram h^ a 

■ dPd' ?n^^" P^lnted^on Its back. Th?s dot c enTn ? n 1 « 
ded 1n the program excerpt given here. 

HoteJ.: The original program was printed In A51 format. To save soace 

Fl!iS2T ^^P^^"*^'^ °" P^g^ In hi applndu: 

Frame numbers In the upper right-hand corner of each frame n- 
dicate the order In which the frames should be read. 
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Kom Ihfig att du skall g& igenom * 

uppglfterna 1 nunanerordning , 

Bbrja overst pfi nSsta si da -> 





22 




Den del * som berh'ttar nSgot om subjekts- 


xxxxxxxxxxxx 


delen. kallas p r e d 1 k a t s d e 1, 




P R E - D I - K A T S - D E L (tonvikt 




pfi KAT) 


• 


Se efter hur ordet stavas och skr1v det h^'r; 




PELLE SPRINGER, - 12 




Denna mening handlar cm PELLE, 




Ordet PELLE ar alltsa 


O 






34 


bri nner 


JOHAN DISKAR, 




Predikatsdelen berSttar n&got om 




subjektsdelen. 




Vllket ord 1 menlngen ovan Hr 




predlkatsdel? 




Svar; 









X 

xxxxxx ^ 


Nh> vi talar, anvh'nder vi mfinga 
o11ka ord. 

Varje ord betyder nSgot. 
Av orden bildar v1 meninqar. 
DU Skriver. 

De tva orden hSr ovan bildar en 


App. 4 (p 3) 


f 

c 


predlkatsdel 


PELLE SPRINQER. 

Det ord son? berSttar nSqct om PELLE 
kallas predlkatsdel. 
Vllket ord Sr p r e d i k a t s d p 1 ? 
Svar: 


23 


subjektsdel 


HUNDEN.SKALLDE'. ' - 

Denna mening ,handlar 1nte om katten 

eller f&geln. 

Den handlar om '~ * 


13 

J 




dlskdr 


BLOI^ORNA VISSNAR. 

Predikatsdelen berSttar nflgot om 

subjsktsdelen. 

Vllket ord ^> predlkatsdel? 

Svar: 


35 
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f 

ft 

0 





mening 
xxxxxx 



FAGELN FLYGER. 

Den hSr menlngen handlar inte om 
hunden eller hSisten, j 
Den handlar om 



springer 



FAGELN FLYGER. 

Det ord som berSttar ndgot om FAGELN 
kallas predikatsdel , 
Vilket ord Sr predikatsdel? 

Svar: f 



hunden 



HUNDFN «;KRLLDE, 



Ordet 



subjektsdel . 



BARNEN SKRATTAR. 
Barnen Sr subjektsdel . 
SKRAtTAR Sr 



FAGELN 

FAGELN FLYGER Sr en mening, men ordet 
FAGELN ensamt Sr inte en 



flyger 



HUNDEN SKALLDE. 

Predikatsdelen berattar nfigot om 
subjektsdelen. 

Vilket ord ar predikatsdel 1 msnlngen 
ovan? 

Svar: 



hunden 



LISA SKRATTADE. 

Den har menlngep handlar Inte om 
PELLE. 

Den handlar om 



predikatsdel 



BARNEN SKRATTAR. 
SKRATTAR Sr predikatsdel : 
BARNEN Sr 





App. 4 (p. 6) 
• 4 


mening 


FA6ELN FLYGER 

Ordet FAGELN sSger vad vi talaV om.* 
For att det skall bTTen mening mfiste 
vi bera'tta nigot om ffigeln. 
viikPt nrd i meninqen berkttar n&got 
om FAGELN? 

Svar: v 






sksllde 


VATTNET FRYSER , 

Ordet FRYSER berSttar nSgot om VATTNET. 
Ordet FRYSER^vSr alltsS 

I? 




16 




LISA SKRATTADE, 

Meningen handlar om USA, 

Orde^ LISA Ur^ v 



subjektsdel 
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38 



En mening har tvfi d^lar 
SUBJEKTSDEL och 
P R E D I K Al T S D £ L.. 
Predikatsdelen berSttar nSgot om 



221 



\ 



\^ f lyger 

\ ■ 

\ 


App. 4 (p 7) 
5 

FAGELN FLYGES. 

Menlngen handler oir. FAGELN. ' ^ 
Den berattar ocksA nSgot 8m FAGELN. 
Dsn berSttar att den 


predikatsdel 
« 


' 27 

PELLE SPRINGER. 

Menlngan handlar m PELLE (subjekts- 
^e 1 ^ • 

Ordet SPRINGER berattar-nAqot om 
subjektsdelen. ^ 

Ordet SPRINGER h'r allts* 


subjektsdel 


17 

FAGELN aVGER. 

Vnket ord ar subjektsdel? 

(Tank ^fter vad menlngen handlar om.) 

■* 

Svar; 


a«bj<5lrtsdelen 


39 

En mening har tvA delar: 

s u b j e k t s d e 1 och 

pVedtkatsdel. 

Vilken del sSger vad menlngen handlar om? 

Svar: 



er|c«' 





App. 4. (p 8) 
6 


flyger 


FA6ELN FLYGER. 

Dessa tvi ord bildar en 




28 


^ predikatsdel 


BARNEN SKRATTAR* 

Meninqen handlar om^BARNEfj^Uubjektsdel ) . 
vnket ord i meninqen ovan berSttar 
nigot cm subjektsdelen? 

Svar: 




18 


ray i <»• 


EVA SPRANG, 

Vllket ord Sr subjektsdel? 

(TSnk efter van meningen handlar om.) 




Svar; ^ 




40 


subjektsdelen 


En mening^har tv4 de1ar: 

subjektsdel och 
predi katsdel, 

Vilken del berSttar nigot om subjektsdelen? 







• mening 


FAGELN FLYGER. • . 
Den hSr menlngen harfdlar omt^ 


App. 4 (p 9) 

7 • „ 


sk rat tar 


BARNEN SKRATTAR. 

Ordet SKRAHAR berSttar nflgcJt om - 
BARNEN (subjektsdelen). 
Ordet SKRATTAR Sr alUsA 


29 


Eva xxxxxxxxxxx 

AXXXXXXXXXXXXXX 

- — 1 — 


1. POJKEN SPRANG FORT, 

2. FLICKAN SPRANG FORTARE, 

Vem^handlar mening nr Tom? 
Svar: 

Vem handlar mening nr 2 om? 
Svar: 


19 


predikatsdelen 


En mening har vanllgen tvA delar. 
Vi kallar den ena /<6l=fi :>ub1ektsdel 
och den andra del en - 

T * 
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d 

*• , f * 


■ > 
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App. 4 (p 10) " 
8 



fSgeln 


betord, som sSger vad meningen 
handlar cm, kallas subjekti- 
d e 1. ... 
PELLE SPRINGER. 

Ordet PELLE ar subjektsdel, eftersom • 
meningen handlar om 




*> 
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predlkatsdel ' 


LAMPAN LY5ER. ' . 
Denna meninq handlar dm. LAMPAN 
Meningen berattar att lampan LYSER. 
Ordet LYSER ar 

— — J 

• 

- * "r^rr 
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1 . Pojken 

2. Flick an 


POOKEN SPRINGER FORT. 

VHket ord *ar subjektsdel 1 

meningen? 

Svar: 
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xxxxxxxxxxxxxxxx 


a 

£n mem'ng ,har vanHgen tvS delar. 
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predikatsdel 



/ 



De kallas 
och 
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Pelle 

. ' ■<? 


, * % 

S u b j e. k t s d e 1 ar ett ' . 
, Ungt ord, 

Vi delar upp det i stavelser: 

SUB - JEKTS - DEL (tonvikt OEKT). 

Se efter hur ordet stavas och skriv 

•det bar: 


App. 4 (p 11) 
9 


predikatsdel 


ISEN SMALTEK. 

Denna mening handlar om ISEN. 
Meningen berattar att isen SMALTER. 
Ordet SMAlTER ar , 
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/ 


pojken 

- 


FLICKAN SPRINGER FORTARE. , 

Vilket ord ar subjektsdel i 

meningen? 

Svar; 
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p 


subjektsdel och 
predikatsdel 


SLUT PA HKFTE 1. 
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subjektsdel 



predikatsdel 



flickan 



Subjektisdelen sager vad meningen 
handlSr om. 

FAGELN FLYGER. 

Denna mening handlar om FA6ELN. 
Qrdet FAGELN ar alltsS 



PELLE SPRINGE?^ j 
Predikatsdelen bercittar nSgot om 
subjektsdelen. 

Vilket ord i fnem'ngen ovan ar 

predikatsdel ? 

Svar: 



Fortsatt nedtill p5 forsta si dan 
i haftet. 



■App. 4 (p 12) 
10 
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App. 4 (p 13) 


subjektsdel 


Subjektsdelen sager vad menlngen 
° handlar om. 

VATTNET FRYSER. 

Denna menlng handlar om VATTNET. 
Ordet^ V/ITTNET ar alltsS 


11 
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springer 



KUSET BRINNER. 

P'redlkatsdelen berattar nSgot om 
^ subjektsdelen. 

^ Vilket ord i menlngen ovan ar 
predlkatsde^ ? 
Svar: 



A. 
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